HERMES: The Hopkins Electronic Resource Management System
This article describes a project undertaken by the Johns Hopkins University libraries to develop a systemwide, Web-based application to facilitate the selection, procurement, implementation, and management of electronic resources and their licenses. The authors detail the history of the project, the function of each of its main application modules, and the various security roles required for administration of the application as well as note similar initiatives and other activities within the library profession to streamline and automate the management of electronic resources.
The Johns Hopkins University (JHU) libraries are nearing completion of a university-wide electronic resource management system. This application, the Hopkins Electronic Resource Management System (HERMES), will provide an easy and time-saving means for patrons to identify and access the electronic resources of JHU libraries, as well as facilitate the process of selecting, purchasing, and managing these resources. The scope of resources managed by HERMES will include all electronic resources to which JHU libraries provide access.
Description and Functional Requirements
Early in the project, the following functional requirements were identified:
- provide a full workflow and approvals process to support the selection, procurement, and implementation of e-resources;
- enable dynamic generation of e-resource information for public display;
- provide automatic notification to appropriate staff of changes of status and scope in e-resource ordering and licensing (including such items as when to renew and action required after a certain number of days);
- provide for link management for e-resources, including the automatic updating of URLs in the backend database, on the campus proxy server, on library Web sites, and so on;
- provide staff with a unified, Web-based means for viewing, updating, reporting, and administering
e-resources, including custom report generation;
- document and maintain information about e-re sources;
- be accessible to all staff and patrons in JHU libraries with appropriate restrictions for use of administrative modules; and
- be interoperable with existing and future systems, including the integrated library system, the campus proxy server, and Web sites of the various campus libraries.
History of the Project
In the spring of 1999, the electronic resources librarian at JHU's Milton S. Eisenhower Library approached the Web application developer and related her need for an easier way to manage the hundreds of links to both licensed and unlicensed electronic resources on the library's Web site. Her problem is probably familiar to anyone who must manage a significant number of Web pages: if a URL changes somewhere on the Web site, the administrator must then track down and update that URL wherever it appears throughout the entire site. This process was tedious and error-prone.
The creation of a simple database-enabled Web application was proposed that would allow the administrator to create a bibliographic record for a particular e-resource and to subject-index the record using multiple subject headings. The public display for this application would pull data directly from the database and would dynamically generate subject-specific lists of e-resources, along with their corresponding URLs. And because the URL, as part of the bibliographic record, would be stored in a single place, any updates to that URL would automatically appear within each of the individual subject lists on the site. In this way, the need for locating and updating multiple instances of the same URL whenever a change to that URL occurred would be eliminated.
An application was constructed and tested that accomplished this task, and a student was hired to perform data entry to populate the backend database. Just as this project was nearing completion, however, another group in the library began developing specifications for an e-resource license tracking database--in other words, an application that would streamline and control the entire e-resources licensing and procurement process from beginning to end. The fact that most integrated library systems do not adequately provide for the workflow and approval processes surrounding e-resource licensing prompted the need for in-house development of such a system. Moreover, there was interest throughout JHU libraries in this project. Most notably, the William H. Welch Medical Library on the JHU East Baltimore campus was interested in participating in the project. With this organizational impetus in place, the earlier project aimed at merely managing links to e-resources was abandoned, and work on an integrated e-resources licensing and management system began.
During the spring 2000 semester, the HERMES committee was formed and several subcommittees were charged with developing specifications for individual modules of the application. These subcommittees were devoted to, among other things, the development of a public display interface, compilation of a list of appropriate subject headings, identification of data elements that should be captured by the backend database, and discussion and preparation of the necessary technical infrastructure. After several months of planning, these subcommittees were dissolved and the results of their work were folded into the work of the central HERMES committee. Planning of the HERMES application, including the design of the backend database and the charting of the entire workflow and approvals process, earnestly began in January 2001. This process required six months of weekly meetings.1 At the end of six months, coding of the application and its backend database began and lasted approximately six months. The Web application developer spent approximately 66 percent of his time coding the HERMES project during this six-month period.
JHU is a decentralized organization, and the libraries of the university are administratively independent. Each library may negotiate and manage its own e-resource licenses, and while the libraries negotiate some licenses on behalf of the entire JHU community, others apply only to specific groups of JHU affiliates. In some cases, costs for e-resources are paid for by a single JHU library, and in other cases, costs are shared by participating libraries. In this way, JHU's needs for e-resource management resemble the needs of a consortium.
The Milton S. Eisenhower Library has invested significant time and resources in providing catalog records for e-resources within the library catalog; over seven thousand e-resources are represented in the JHU library catalog. Data elements required for HERMES include bibliographic data already included in the JHU catalog, as well as many other data elements not captured in a traditional integrated library system. To avoid duplication of data entry, HERMES must automatically capture appropriate data from the JHU library catalog. In order to accommodate the goal of interoperability with existing and future systems, an XML document type definition (DTD) was formulated to serve as HERMES's bibliographic data import format. The current JHU library catalog, a future catalog system, or any other data source with bibliographic metadata about electronic resources may be used to populate HERMES, as long as extracts can be manipulated into the appropriate XML format. HERMES may also be used as a stand-alone system; it allows for manual data entry of all elements if a load from another data source is not available or not feasible.
While it was obviously important for the HERMES system to accommodate the workflow and management needs of the JHU libraries, the developer also sought to make the system useful for other libraries with different approaches to e-resource management. The system is designed to accommodate a variety of approaches to use. With minimal modification of the application code, new workflow steps can be inserted or other similar changes made. Even without code modification, customization for local needs can be accommodated through the specific functional roles available for assignment to users of the system. These functional roles represent individual tasks and responsibilities, such as selection, acquisition, and cataloging. At JHU, most staff will have only one or two roles within HERMES, while at a library with a smaller staff or a more centralized e-resource management workflow, a single staff person might be assigned more roles and granted greater rights and responsibilities.
Brief Description of Main Modules
In order to accommodate each step of the e-resource acquisition and management workflow identified in the planning stages, HERMES was developed as a modular application.
This module provides authentication of administrative users against a central lightweight directory access protocol (LDAP) authentication store.2 This module merely refers to an externally administered, HERMES-independent LDAP directory to determine whether or not an administrative user truly is who he or she claims to be. If the user is successfully authenticated, this module begins a session based upon the authenticated username. Such information as username/password generation and password resetting are all handled by the outside LDAP service and need not be administered separately by the HERMES application. This reliance on an external LDAP directory obviously requires that the implementing institution have an LDAP directory available for authentication of staff users.
This module checks an authenticated username against an internal authorization table to ensure that the user is authorized for access to the administrative modules of the application. The contents of this internal authorization table provide detailed information with respect to the functions and modules the authenticated user has authorization to access.
This module allows selectors to initiate and track the selection process. At any time selectors have the ability to view the status of their open items within the overall HERMES workflow--that is, they can quickly determine what stage an item is in with respect to procurement. This offers obvious advantages over a selection process based on paper forms.
This module governs the acquisitions process for ordering, receiving, renewing, and decommissioning of e-resources. It is the module where license management occurs. Within this module, license terms are defined as actionable elements to allow the system to appropriately allow or deny access to specific user groups. Furthermore, the actual legal license documents may be scanned and uploaded for easy and secure storage and retrieval.
This module is perhaps the most complex and sophisticated module in the HERMES application. Since JHU continues to invest effort in traditional cataloging of e-resources, the catalog interface was developed to accommodate automated interaction with the library management system (or any other source with information about e-resource subscriptions). Data in HERMES can be synchronized with data from another resource such that, when a data field in a record in the external resource changes, so too will its value in the corresponding record in the HERMES database. In this way, such things as titles or imprints can be maintained within the external resource, with changes flowing through on a scheduled basis to HERMES. Each HERMES field may be configured to accept changes coming through the catalog interface or to retain existing HERMES data; this allows HERMES to serve as the authoritative source for some data elements, while the external resource serves as the authoritative source for other fields.
Technically, this process is accomplished through use of XML technologies. An XML DTD was created reflecting the structure of the records held in the HERMES database tables.3 Data need only be extracted from the external resource and formatted in accordance with this DTD. The resulting XML packet is then placed somewhere on the Web in a location accessible in programmatic fashion by the HERMES application. On a scheduled basis, HERMES grabs this XML-formatted extract via HTTP and pulls it into the HERMES database, making required updates to existing records along the way. Thus, select data in HERMES can be synchronized with the contents of the local library catalog or external resource.
This module controls the final catalog entries of e-resources. It also provides an authorized staff member the ability to manage the thesaurus of subject terms used to index records in the backend database.4 Subject-indexing of e-resources is ultimately performed within this module.
Library Computing Services
This module allows the administration of a lookup table of IP addresses mapped to a standard list of access locations, and a workflow process for implementing e-resources that require special attention from library computing staff (for example, networked CD-ROMs).
This module controls the logic and display of what public clients actually see over the Web.5 Provision for search and retrieval of data about electronic resources is made here. Specifically, the client has the ability both to browse and search the portion of the collection of e-resources flagged as appropriate for public display from within this module. This module allows participating libraries to dynamically pass in header and footer files resulting in output to the client, resembling, as closely as possible, the look and feel of the local library Web site.
This module enables authorized administrators to search the entire HERMES database for records matching select criteria.
This module provides reports and statistics for view by authorized staff.
This module runs on a scheduled basis and automatically notifies the appropriate staff via e-mail of relevant changes in such items as content status and licensing.
Automated Subject Indexing
One of the interesting features of the catalog interface is its ability to automatically subject index records based on their assigned Library of Congress (LC) or MESH subject headings. Catalogers in the HERMES application have the facility to map incoming LC and MESH headings to counterparts in the HERMES subject schema. Once a given LC or MESH mapping has thus been created, any future records entering the HERMES system via the catalog interface will automatically be mapped to the corresponding HERMES subject heading, thereby ensuring that the record is accurately subject-indexed and easily retrievable via the public display interface.
So, for instance, suppose an incoming record has the following LC heading:
The catalog interface determines whether it has previously "seen" this particular subject heading before. If not, it inserts it into a backend lookup table and alerts the appropriate cataloger, informing him or her of the existence of a new LC heading that requires a mapping to the HERMES subject schema. However, if the catalog interface determines that it has indeed seen this LC heading before, and if a cataloger has supplied a mapping into the HERMES subject schema for it, then the incoming record is automatically mapped to the HERMES heading--no intervention on the part of a cataloger is required. And this process will occur for any and all incoming records via the catalog interface that are subject indexed by that same LC heading.
Application Roles and Administrative Groups
During the design phase of the project several administrative roles were identified that would be required for the secure administration of the application. The following are the roles and groups that were identified.
This group--consisting of faculty, students, researchers, university staff, and the general public--can search, retrieve, and display datasets from the backend database via the publicly accessible search interface.
Super-Users function as system administrators of the HERMES application. This role has access to all functions, and may add administrative users, assign roles, modify several backend lookup tables, and monitor application errors.
This role allows an authorized individual to initiate the resource procurement process on behalf of a participating library and track the item's workflow progress. Each selector is assigned one or more budget codes that he or she may suggest for use in purchasing selected electronic resources.
The super-selector flags selected items for approval or disapproval, and thus for passage to the next phase of the procurement process, and decides what will actually be purchased from among the pool of selected items. The super-selector assigns budget codes to selectors. There must be at least one super-selector per participating library, and the super-selector has control over the selections proposed for that particular library.
Acquisitions administrators are responsible for procuring items and for tracking them throughout the procurement process. This role also administers and maintains license data, including scanning and upload of digital copies of the licenses themselves. Finally, acquisitions administrators maintain the table of budget codes that super-selectors may assign to selectors.
This role is responsible for properly cataloging items in HERMES, including assignment of subject headings. If bibliographic data for e-resources is imported from another source such as the library's catalog, catalog administrators are responsible for completing records with HERMES-specific data and for administering the automatic subject-indexing module.
Library Computing Services Administrators
This role is responsible for approving the purchase of electronic resources with special computing requirements--for example, CD-ROMs and other standalone databases.
Public Display Administrators
This role is responsible for deciding, out of the domain of purchased and cataloged items, which ones appear on the publicly accessible search interface. This role can alter the bibliographic record of an item at any point, ensuring that the record for the item is sufficiently complete and fit for public viewing.
Certain members of the library staff who might not otherwise fit into any of the other administrative roles can log on to HERMES and query the database for licensing information. Staff assigned this role include members of interlibrary loan and reserves departments. This role can only query and view records in the database--it has no insert, update, or delete privileges on any of the data.
As of this writing, the HERMES application is comprised of thirty-six database tables and 130 files containing 12,007 lines of program code. The backend database is currently Microsoft SQL Server 7.0, but the application was written in such a way that any relational database management system (RDBMS) supporting transactions and subselects could be used instead. Due to its simplicity, power, scalability, and value as a Web application development platform, Macromedia's ColdFusion was chosen as the middleware platform.6 The ColdFusion developer community is large and very open with its custom libraries (in ColdFusion-speak, "tags"), hence a handful of custom libraries were also incorporated into the HERMES application.7 Due to the unacceptable performance of existing XML parsers, a portion of the catalog interface was written in Python, a scripting language capable of extremely fast text manipulation.
Similar Projects and Related Initiatives
Many other libraries have recognized the need for organizing the e-resource management process, and several libraries have developed automated systems similar to HERMES. A small group of librarians interested in the problems and issues with e-resource management met at the 2001 American Library Association (ALA) Annual Conference in San Francisco. The group organized a larger meeting at the 2002 ALA Midwinter Meeting in New Orleans; HERMES committee member Nathan Robertson was among several meeting participants who gave presentations on locally developed e-resource management systems.8
Many attendees at the 2002 ALA Midwinter Meeting expressed interest in developing a metadata standard for e-resources to facilitate their management; this standard would include not only descriptive metadata, but also metadata to accommodate licensing details, access restrictions, and administrative management. In May 2002 NISO hosted a workshop in conjunction with the DLF Forum in Chicago for a group of librarians and vendors to explore issues and ideas regarding e-resource management and e-resource metadata. A follow-up meeting at the 2002 ALA Annual Conference in Atlanta was well attended.9
In September 2002 the Digital Library Federation took on official sponsorship of the Electronic Resource Management Initiative project.10 There is clearly growing interest from the library community in the issues of streamlining e-resource management.
Post-Coding History and Current Status
As of this writing, the HERMES application is undergoing intensive testing, and bugs are being resolved. Once testing is complete, the backend database will be populated, initially with an XML-formatted extract from the local library catalog. However, because the initial data dump will only be comprised of bibliographic rather than licensing data, a long period of manually entering licensing data and fleshing out the bibliographic records will necessarily follow. Then, with complete bibliographic and license records in place, they will have to be mapped, one to the other--that is to say, each bibliographic resource record must be mapped to its corresponding license. Only once this process is complete will HERMES be ready to go live. It is hoped this can be accomplished sometime during the spring 2003 semester. Once live, all requests for new e-resources and all e-resources licensing will use HERMES as the primary tool for managing these complex processes.
After the current testing and debugging of HERMES is complete, the Sheridan Libraries of the Johns Hopkins University plan to release HERMES as an open source software product for other libraries to customize and use.
The authors wish to express their sincere thanks to everyone at JHU involved in the HERMES project, particularly the regular members of the HERMES committee: Lori Foulke, Rebecca Graham, Dawn Hale, David James, Pat Lovett, Ginny Massey-Burzio, David Reynolds, Mary Ann Urka, and Sue Woodson.
References and Notes
1. This planning was strongly influenced by several examples of similar, ongoing projects across North America. Adam Chandler (Cornell) and Tim Jewell (University of Washington) maintain a useful Web site listing these efforts. Accessed Dec. 13, 2002, www.library.cornell.edu/cts/elicensestudy.
2. For more information about LDAP, see www.ldapzone.com, accessed Dec. 13, 2002.
<?xml version="1.0" encoding="UTF-8"?>
<!ELEMENT alternatetitle (#PCDATA)>
<!ELEMENT catalogid (#PCDATA)>
<!ELEMENT collectioncode (#PCDATA)>
<!ELEMENT contributors (#PCDATA)>
<!ELEMENT coverage (url?, collectioncode)>
<!ELEMENT coverageurl (coverage+)>
<!ELEMENT edition (#PCDATA)>
<!ELEMENT eissn (#PCDATA)>
<!ELEMENT formertitle (#PCDATA)>
<!ELEMENT imprint (#PCDATA)>
<!ELEMENT isbn (#PCDATA)>
<!ELEMENT issn (#PCDATA)>
<!ELEMENT language (#PCDATA)>
<!ELEMENT lcmeshheading (#PCDATA | lcmeshheadingflag)*>
<!ELEMENT lcmeshheadingflag (#PCDATA)>
<!ELEMENT libraryid (#PCDATA)>
<!ELEMENT newtitle (#PCDATA)>
<!ELEMENT printequivalent (#PCDATA)>
<!ELEMENT record (catalogid, title?, contributors?, alternatetitle?, contributors?, imprint?, edition?, issn?, formertitle?, printequivalent?, eissn?, formertitle?, specialviewinginstructions?, newtitle?, specialviewinginstructions?, formertitle?, printequivalent?, isbn?, newtitle?, issn?, formertitle?, printequivalent?, specialviewinginstructions?, language?, subjectheadings?, libraryid?, typeofresourceid?, coverageurl)>
<!ELEMENT recordset (record+)>
<!ELEMENT specialviewinginstructions (#PCDATA)>
<!ELEMENT subjectheadings (lcmeshheading+)>
<!ELEMENT title (#PCDATA)>
<!ELEMENT typeofresourceid (#PCDATA)>
<!ELEMENT url (#PCDATA)>
4. Mark Cyzyk, "Recursive Custom Tags," ColdFusion Developer's Journal 2, no. 10 (Oct. 2000). The heart of the backend portion of the subject module consists of a single table representing a linked-list data structure of subject terms as well as program code that recursively interacts with the data in this table.
5. The look and feel of the search interface used in this module as well as the overall logic of how searches are done in this module were based upon the HERMES Public Display Subcommittee's long, hard scrutiny of the very fine example provided by the California Digital Library. Accessed Dec. 13, 2002, www.cdlib.org.
6. Mark Cyzyk, "Script Junkie: ColdFusion Markup Language," Web Techniques 5, no. 8 (Aug. 2000). See for discussion and examples of why ColdFusion is superior in many respects to other Web application platforms. Accessed Dec. 13, 2002, www.webtechniques.com/archives/2000/08/junk.
8. A report on this meeting is posted to www.library.cornell.edu/cts/elicensestudy/alamidwinter2002.htm, accessed Dec. 13, 2002.
9. A report on this meeting is posted to www.library.cornell.edu/cts/elicensestudy/alaannual2002/home.htm, accessed Dec. 13, 2002.
10. This initiative is outlined at www.diglib.org/standards/dlf-erm02.htm, accessed Jan. 9, 2003.
Mark Cyzyk ( firstname.lastname@example.org) is the Web Architect at Johns Hopkins University, Baltimore, Maryland. Nathan D. M. Robertson ( email@example.com) is a Database Analyst/Programmer for the Milton S. Eisenhower Library of the Johns Hopkins University.