Subject Web Page Management without HTML Coding: Two Approaches

This article describes two examples using relational databases to streamline the creation and management of active, Web-based subject bibliographies. Before the database approach, library staff expended considerable time and effort compiling subject Web-resource pages to guide users to high-quality resources. The process of producing subject guides was tedious, repetitive, and labor intensive, requiring librarians to become proficient at the intricate task of Web-page creation. Since identical resources, descriptions, and links frequently appear on several different pages, there was considerable duplication of information. Wesleyan University and the Tri-College Consortium each, independently, sought to solve this problem by creating a database of resource information and a process for mapping guide pages. This report compares their different approaches, contrasting in-house versus outsourcing approaches, an independent database versus one built from OPAC, and open source versus proprietary software.

Wesleyan University Library (WUL) in Middletown, Connecticut, and the Tri-College Consortium (TCC) of Bryn Mawr, Haverford, and Swarthmore Colleges near Philadelphia faced a challenge common to many libraries in their need to create subject-specific Web pages for library users. Creation of these pages by appropriate subject specialists required that they either learn to manipulate HTML coding, or to use a Web composition software program. Although the latter option is easier than direct coding, it still requires mastery of a new software application for the sole purpose, in all likelihood, of producing these subject guides. The subject specialist must spend considerable time formatting pages, keying descriptive data about the resources, and troubleshooting unexpected problems with online displays. Not all librarians are equally comfortable with writing Web pages, and individual comfort levels discourage and delay both creation and timely updates of existing pages. A small college library staff is simply unable to develop and maintain Web research guides in this complicated manner.

Many of the same resources, such as major reference works, indexes, journals, and meta Web sites are duplicated on different subject pages. As the Web resources change their URLs, coverage, or other characteristics, each occurrence of the identical data in different pages needs first to be located, and then appropriately updated. Subject guide pages tend to quickly become outdated. Since several different librarians write the pages, they may even be unaware of updates needed, and of descriptions previously written by their colleagues. This factor contributes to a considerable duplication of effort.

The goal of WUL and TCC was to find efficient ways for librarians to create and update subject research guides. Both institutions addressed this issue independently in 2000, unaware of the other's involvement with it. In January 2001, through a chance discussion, it was discovered that both institutions were working toward similar solutions to the same problem, although with very different approaches.

The Two Approaches

The solutions employed by WUL and TCC are based on the creation of relational databases for tracking resources and building page content to enable the dynamic generation of research guides. The goal is to enable librarians, instead of writing separate and static research guides, to quickly enter or select resources and arrange them on a page through a simple staff interface. Updates of URLs and other resource information can be made once on the database record. Since page displays are created dynamically from the database, the updates take effect immediately on all relevant pages. Outdated resources can be quickly deleted from the database and thus from the pages. It is also possible to compile electronic guides for users on demand by allowing them to search the database itself.

WUL built their solution on an existing database of electronic resources, separate from their library catalog. Librarians enter additional resource data to it as needed for new pages. A set of resource categories was agreed on, so that librarians can create a page for a new subject area by selecting resources from the database to be displayed in each appropriate category, or by leaving that category blank. Records in the database can be updated by another simple interface. WUL built their application using open source software and in-house programming skills.

TCC has a policy of entering records for all online resources, both subscription and free, quality resources, in their shared catalog. They therefore created their resources database by extracting the appropriate records and reloading them into a secondary database. New resources and updates are accomplished by adding to or correcting the library catalog and performing regular reloads of the resources database. As with WUL, resource categories were agreed upon, and staff can create a new page by selecting resources from the database and assigning categories. The TCC interface allows more freedom for arranging page structure. This application was built using commercial software (MS SQL and ColdFusion), and by outsourcing the initial development. Ongoing enhancements and maintenance are now done in-house.

A Local, Open Source Solution

The WUL approach was to create a subject database totally independent of the online catalog. For a few years prior to the beginning of the subject guides project, WUL had already begun to maintain Web resources in MySQL with PHP as the scripting language in order to generate Web pages on the fly. Lists of subscribed databases and journals were already available in the MySQL databases. The CTW Library Consortium (Connecticut College, Trinity College, and Wesleyan University) had made a collective decision not to catalog free Internet resources in the CTW online catalog. In addition, the three institutions had not planned to collaborate on the production of library research guides. WUL's solution was to develop a subject Web bibliography as a local database based upon the preexisting data in the MySQL databases.

Before the dynamic subject guides project began, WUL librarians already used a standard template to maintain static subject guides. The creation and maintenance of subject guides was uneven, however, because librarians had other pressing professional responsibilities, and because there was a wide variation in individual librarian's skills in using Web authoring software efficiently. Some of the subject guides were updated in a timely manner while others were not.

WUL chose to maintain the look and feel of the existing static subject pages, and librarians were asked to develop a standardized set of categories and order of presentation in which to list the selected resources. Librarians have the flexibility to list resources using as many or as few of the categories as they think appropriate to their specific subject page. Even when the same database is listed in several subject pages, every librarian also has the freedom to write his or her own descriptive annotations to inform users why a particular resource is valuable for a particular subject.

The WUL subject pages are designed to direct users to significant electronic as well as print resources available in the local reference collection. Links to general research tutorials are included on each subject resource guide page to remind users of standard research methods (see figure 1).

Figure 1

Headers, opening narrative, and links on the right side of each page are also standard to each subject guide. Changes to standard text, such as an announcement of a new service, or changing the URL for a tutorial page, need be executed only once for all subject guides. The lower half of each guide lists, in a standard presentation, the categories of resources available. If users choose to print the guide, they will see the annotated bibliography in a special print format. All WUL subject guides look very similar, but they contain unique descriptions written by a librarian who is a specialist in the respective discipline.

As shown in figure 2, librarians at WUL use a staff interface to construct subject guides. Pull-down lists are available to add, modify, or delete a subscribed Web resource to the guide (see figure 3). Librarians do not enter the URLs since they are included in the master list of subscribed indexes and databases; the existing record already provides the title and URL. This feature allows for the dynamic update of all subject Web pages whenever a URL is changed for a particular resource to which WUL subscribes. As noted above, the librarians may add their own descriptive information appropriate to each guide.

Figure 2

Figure 3

Librarians may also assign a priority ranking to each resource through the Display Rank function, which forces the most important resources to the top of a particular listing. Figure 4 shows a separate Web form that is specially formatted to add print resources. The librarian simply types the title and call number of the hardcopy reference item and bypasses the URL field. As with electronic resources, the librarian may decide to add a unique annotation and a display ranking for each source item. The same form, as demonstrated in figure 5, may also be used to add free Web resources, this time using the URL field. The data entered into these forms is stored in a database, and the guide page display for library users is built by retrieval from that database.

Figure 4

Figure 5

The WUL subject resource database was developed and created locally by student programmers whose work was supervised by a permanent staff member. With an annual student budget of $5,800, at least two undergraduate students are hired each year to create and maintain various database applications in the library. WUL makes a point of hiring one junior to work with a senior who has more experience and who, therefore, can train the new student in the application before graduation. The team approach allows students to pass on knowledge of the design for ongoing maintenance, and to provide reliable support. The employment of students for this project automatically provides the library with user input on design issues, while librarians are consulted on more formal issues. Since 1999, the undergraduate student programmers have also created a journal locator, dynamic lists of indexes and databases, and other internal databases to track such things as the library's liaison activities and instruction programs.

The subject resource database was created over a period of three months. Reference librarians met several times to agree on the categories and the look of the design. A student programmer was present at one of the librarian meetings to better understand the needs for some of the features. One of the side benefits for the library is the valuable input that comes from students' perspectives. Students, on the other hand, have the satisfaction of seeing their creations used by their fellow students.

The WUL system was developed using the open source philosophy. It is mounted on an Apache Web server, using Red Hat Linux, MySQL, and PHP3, and is hosted and maintained locally. The public view of the system is available at www.wesleyan.edu:9092/libr/php/subjects.

A Commercial, Outsourced Solution

TCC shares a single integrated online catalog and has worked, since its inception, to have a single shared bibliographic record for each item in OPAC. When the decision was made to catalog Web resources, the same philosophy was applied÷one resource, one shared bibliographic record. TCC OPAC includes cataloging for subscribed Web resources, individual titles of aggregate full-text subscriptions, and selected free Web resources. Catalog records are contributed by the catalogers and reference librarians from the three member institutions. TCC librarians decided that Web publications would be represented in OPAC by separate records, instead of information appended to the records for the printed versions. Two different publishers of the same online title or resource also require two separate bibliographic records. These practices contributed to the overall solution decided upon for the Web guides issue.

Although the TCC libraries' catalog has been merged for more than a decade, Web page design has been unique to each library. Each institution developed its own set of Web information pages for library users, and created its own set of subject, research, and course guides. In 1999 TCC decided to make the development of a single standard for Web resources a high priority because sharing this work offered an excellent opportunity to reduce duplicate effort, share expertise, and better serve the students. TCC intended, at first, to share static Web guides by creating a shared template and a single consortial guide for each subject area. Staff costs for the building and maintenance of the Web guides became an issue, however, as the librarians worked on the shared development project. The need to free subject specialists from tedious, repetitive work, as previously mentioned, and discover ways to update URLs automatically became obvious. An article in Information Technology and Libraries brought the active Web pages option to the attention of TCC librarians. 1

Because of its shared OPAC and ongoing commitment to cataloging Web resources, it was natural for TCC to build dynamic Web guides using OPAC as the source for data. The TCC plan, simply stated, was to outsource development of the Web guide publishing system, and to use existing catalog records as the data source. TCC OPAC software has a proprietary data structure so that the bibliographical records could not be accessed directly. The librarians, therefore, made the following decisions:

• standard catalog procedures would be used to create and maintain resource information in OPAC;
• OPAC data would be batch-loaded on a regular basis into an SQL database in which appropriate fields would be selected for use in the guides;
• the SQL database would be used to populate the Web guides; and
• an easy interface would be developed to assist librarians in the selection and arrangement of resources in the guides.

  Since TCC librarians decided to share the work of Web guide creation, they made the further decisions that:

• each subject Web guide would include resources available in any of the libraries;
• guides would include all relevant resources, whether online or print; and
• staff from the three colleges could share in the creation and maintenance of any particular guide.

The Public Interface

The public interface to the TCC Web guides emphasizes their consortial nature. Individual library availability is indicated with circled letters that serve as links, and for print resources, in the location statements (see figures 6 and 7). Resources are divided into a set of agreed-upon categories, and may be further divided into subcategories input and organized by the guide creator.

Figure 6

Figure 7

The Staff Interface

The first step in the creation of a new TCC Web guide is to search for relevant resources in the SQL database, as shown in figure 8. This database currently contains approximately 29,000 records. A resource list (figure 9) is retrieved from that search, from which the guide author selects items for inclusion in the subject guide. If a particular resource is not in the database, the author can request that it be loaded in the next weekly batch. The records to be copied from the library catalog to the SQL database are coded; if a record does not exist, it is cataloged. When the record is in the SQL database, the resource can be added to the guide. Although this process is indirect, it has the advantage of making the resource available in OPAC.

Figure 8

Figure 9

Once resources have been selected÷a process that can involve several searches÷the author assigns a category to each resource (figure 10). Guides (figure 11) are displayed to the staff member in an abbreviated form during most of the construction process. Editing options are shown above the display. The Preview Guide function displays the full public version for review.

Figure 10

Figure 11

Figure 12 shows how the author may also add free-form subcategories to any category that will be unique to the particular guide. Categories, subcategories, and resources can be rearranged as desired by the author (figure 13 shows reordering of resources). Resources may also be deleted as necessary. Subcategories and categories containing no resources are not displayed in the public view. The author may also add text annotations to subdivision labels and to individual resources, or may accept existing resource descriptions maintained in and imported from OPAC.

Figure 12

Figure 13

The subject guides now available serve as lists of the most valuable resources on a discipline or specific topic. Research guides, currently in development, are intended to be more instructive in nature; the interface allows more space for extensive annotation. A search interface (see figure 14) is also available that allows users to search the SQL database using either their own terminology, or selecting from a preset list of subject headings. These searches may be limited by format, such as electronic journals, electronic books, or indexes.

Figure 14

The design and construction of the TCC Web guides was overseen by a team of representatives from all three colleges: Cecelia Buchanan, Tri-College instruc­ tional technology coordinator; Berry Chamness, Bryn Mawr College head of cataloging; Mark Colvson, Bryn Mawr College head of reference services; Mary Lynn Morris, Haverford College digital services librarian; Tammy Rabideau, Swarthmore College digital services coordinator; and one of the authors, Linda Bills.

The construction of the Web guides was outsourced to a local company, TechSense Solutions (www.techsense.
com). TCC libraries did not have a staff member with either sufficient expertise with Web-based database applications or the time necessary for complex development, including downloading and overlaying data, designing the database structure, developing staff interfaces for building the guides, and designing patron views.

Planning was done by a committee of six members, one representing the three computing centers. After deciding on outsourcing, the group interviewed two local companies and selected TechSense. Working with TechSense on the design took two initial meetings, with follow-up examination and redirection as the design emerged. TCC paid TechSense $26,900 for the product and an additional $14,400 to train five staff members in ColdFusion and the specific application so that changes and maintenance could be accomplished in-house. These fees were covered as part of a grant from the Mellon Foundation.

The Web-guide project soon became a learning experience in outsourcing development for TCC. The staff discovered that much more time should have been devoted at the beginning of the project to considering all of the design details, and to thinking carefully about how the software, and particularly the staff interface, would be used. Initial design discussions with the database developer appeared to be thorough, but when the first prototype was examined, it soon became obvious that important design issues had, in fact, been overlooked.

The project was scheduled to have taken six months to complete. Due to changes in the design and difficulties the contractor found in using MARC records to create the load, it took a full year to complete the project. TCC's ongoing relationship with the vendor allows for additional assistance with future development, and for the maintenance of off-site hosting. A Web specialist has been hired to continue development of this project and other consortium-wide Web endeavors. This person has spent the majority of his time in the last eight months developing Web guides to meet specific needs of the reference staff. It is the feeling of the committee that many of these needs could have been met in the design stage if more time had been devoted to initial planning. However, until the committee saw the Web guides in operation, the newness of the concept made it difficult to imagine the finer issues of how wanted the guides to work.

The TCC system was built with SQL, ColdFusion software, and a standard Web server. The system is presently hosted off-site with an Internet service company, but soon will be brought in-house. The public interface is available at www.tricollege.org.

Similar Goals, Different Approaches

The following points highlight the differences between WUL and TCC in the development and management of subject Web pages:

• Open source versus commercial software. WUL has positioned itself to share and adopt other open source applications by the development of local expertise over the last few years. MySQL and PHP are natural choices for the subject resource database, while the TCC model found the use of a commercial software program more suitable. Staff at TCC have learned to use ColdFusion as a result of this project, although continued development and maintenance might be subcontracted if desired.
  
  
• Local versus outsourced development. Since WUL had already developed its local ability to create and manage dynamic Web pages, it was logical for them to create the new subject resource databases locally. TCC, on the other hand, found that the most expedient means to achieve its goals was to outsource its development to a company with both database and ColdFusion expertise. TCC librarians believed that this approach would optimize the design both for functionality and speed. The application was complex; it involved the reformatting of MARC records, overlaying of records, dealing with deleted resources, and creating an interface that was easy to use and which provided the librarians with maximum flexibility. The vendor provided excellent program documentation and training, and will be available when needed in the future. The process was especially valuable in teaching TCC librarians how to manage complex projects with an outside vendor.
  
  
• Database creation. The WUL decision to create a totally separate database of subject resources instead of one closely allied to the CTW online system was made for a variety of reasons. WUL had never maintained records for free Internet resources in OPAC. Bibliographic records for subscribed databases were already stored in a separate database that became an easily used point of departure for the bibliographical research guides. Library staff are able to add print or free Internet resources to that database as a function of guide production. TCC, on the other hand, has established the policy of making the shared OPAC the preferred location for all resource information and record keeping. It was obvious, therefore, that an efficient system of Web guides had to function as a direct outgrowth of OPAC. Delays created by the need for batch uploads are offset by the advantages of a single master catalog for all resources.
  
  
• Statistics. WUL, from the beginning, incorporated a statistics-gathering feature into the database design. This feature has made it possible for librarians to document the frequency with which guides are used, and whether or not suggested links are actually being accessed by library users. WUL librarians had frequently asked whether or not the time spent in the creation and maintenance of Web bibliographies was, indeed, justified by the amount of use they received. By generating and updating subject guides dynamically, the entire operation has become efficient enough that justification is no longer needed. The usage statistics collected so far have clearly demonstrated that library users do, in fact, follow the source recommendations on the subject Web guides to either free or subscribed Internet resources. Although TCC application does not, at this time, allow for statistics collection, it is a feature that may be included in a later software enhancement.

Conclusions

The two very different approaches taken by WUL and TCC to address subject Web page management have both achieved satisfactory results. Library staff, at both institutions, are now able to create structured subject Web guides without the use of complicated, time-consuming Web-authoring software; resource annotations can now quickly be posted, and pages can be updated, corrected, or deleted easily from a single file. Equally important, any addition or modification to commonly used commercial databases can be updated simultaneously across many subject bibliographies.

The uniform templates used by both of these systems have resulted, to some extent, in the loss of individual writing style for the presentation of the library resources. Both systems have circumvented this problem by giving the librarians flexibility to write their own resource annotations, to display the resources in order of importance, and to use as many or as few categories as they believe necessary. The TCC system has moved from using unique guide formats for the three institutions to a single format used for all the libraries in the consortium. In the cases of both WUL and TCC, the standardization of guide formats has proven beneficial for library users.

Reference

   1. Kristin Antelman, "Getting Out of the HTML Business: The Database-Driven Web Site Solution," Information Technology and Libraries 18, no. 4 (Dec. 1999): 176-81.

   Linda Bills( lbills@brynmawr.edu) is Special Projects Coordinator for the Tri-College Consortium (Bryn Mawr, Haverford, Swarthmore), Haverford, Pennsylvania. Rachel J. Cheng ( rcheng@emich.edu) is University Librarian, Eastern Michigan University, Ypsilanti. Alan J. Nathanson ( anathanson@wesleyan.edu) is Bibliographer/Reference Librarian at Wesleyan University, Middletown, Connecticut.