Roles in Digital Reference
Over the course of the past five years, researchers and practitioners have demonstrated that digital reference services can indeed work well and have developed much of the necessary technology. The next step we must take is to figure out how to optimize the design and operation of our services. A useful step in this direction is the development of consensus models that describe the digital reference process. The authors have developed a model that describes the various roles played by participants in this process and the ways in which those roles interact. This model is illustrated by several case studies: the Internet Public Library, the Saskatchewan Provincial Library, and the Virtual Reference Desk network. The authors hope that the model will facilitate further research by providing a framework and terminology for discussion about the digital reference process. Furthermore, it may be useful to practitioners in the field who are engaged in designing and evaluating policies and procedures for digital reference.
Over the course of the past five years, the nascent field of digital reference has matured greatly. Early on, the primary goal was to demonstrate that online reference services could actually work, and to develop the technology necessary to do so. This initial goal has been achieved. There is now no doubt that digital reference can be very effective, and furthermore that services can work collaboratively and on a large scale. Several organizations, including our own project, have developed successful software tools for digital reference work.
The next step for digital reference research, as with any field in which the demonstration phase has passed, is to figure out how to optimize the design and operation of our services. In order to do this, it is necessary to develop consensus models that describe the digital reference process. Such models will serve as a common basis for discussion, and will also provide the degree of abstraction necessary for high-level reasoning about any system. They must be sufficiently generic to apply to the wide range of procedures that are used by various projects around the world, and at the same time specific enough to serve as the basis for concrete analysis and experimentation.
There are many facets to digital reference work that are amenable to modeling. In this paper, we will consider the roles that are played by the participants in the process. These roles define both the interaction between the various participants and their functions with respect to the operation of the service. Thus, they provide a good basis for future discussion and for the modeling of other facets of the field. One productive way to use these roles is as a basis for organizing the kinds of policy decisions that are necessary in order to develop a digital reference service.
Many of the decisions that go into building a digital reference service are ones with which librarians are already familiar: developing an efficient staffing schedule, offering a tiered versus nontiered service, setting up a system for keeping usage statistics, and so on. Librarians, in particular, have developed a great deal of experience over the years in creating reference service models that match the available resources and the needs of their community. Making the transition to the digital environment involves building on this expertise, while at the same time keeping in mind that the new environment imposes radically different conditions and raises important new questions.
One good way to consider these new constraints is to focus on the various roles played by participants in the digital reference process. These roles are familiar to those involved in the traditional reference process. Each has some new twists imposed by the digital environment and should not be considered to be exclusive. However, the separation helps us illustrate some of the issues that ought to be considered.
The fundamental role in the reference process is the asking of questions. Whether these individuals are referred to as "users,""customers,"or "patrons"in the digital world, they require the same level of service as if they had walked in through a door, though providing much less information about themselves. In the classical model of reference, the patron interacts with a librarian in person or over the telephone, and can be thoroughly interviewed and ascertained to be a member of the community to be served. The librarian can respond to visual or voice cues - young or old, happy smile or disgruntled frown. The answer is delivered in person, thank-yous are duly received, and the transaction is promptly forgotten save as an entry in a log.
The digital librarian, by contrast, receives a textual message by e-mail, Web, or chat. The librarian must determine, with a limited amount of context, who the patron is and what question they are really trying to ask. The patron may be located anywhere in the world, and may be of any age, gender, and profession. Even if asked directly, they may be reluctant to provide such information. Lacking such auxiliary cues, it is imperative that the "reference interview" represented by a Web form, e-mail template, or chat script be carefully designed to elicit enough information that the reference librarians can answer the patron's real question. One very useful technique is to ask the patrons how they are planning to use the information. In our experience, the answer to this question is invaluable in figuring out what the patron really wants to ask.
Once the answer is provided and sent back to the patron, further questions arise. The systems that allow for digital communication make it easy to keep a record of the entire interaction with the patron. Such records can be invaluable for subsequent self-evaluation, as well as for generating archives of frequently asked questions and answers. However, this brings up concerns about privacy and intellectual property rights (which are extensive enough to be beyond the scope of this paper).
Given all of the problems of patron interaction in the digital world, there are at least a few advantages. First of all, the patrons have much greater access to digital services than to physically located ones. Depending upon its policy, a digital reference service can receive questions from anywhere in the world at any time. If a question cannot be answered locally, it can be forwarded to someone who can answer it. This gives the patron a better likelihood of getting a good quality answer. Use of the ever-growing world of digital resources allows the patron to view the same sources that the question-answerer used. In addition, our experience shows that patrons are more likely to ask questions that may be personally embarrassing to them via e-mail than they are in person.
As we have noted, there are many characteristics of digital communication which serve to obstruct the flow of information between the person who asks a question and the person who answers it. On a large scale, these same characteristics are responsible for muddying the flow of incoming questions. The questions arriving at the in-box of a digital reference service are inevitably mixed with various kinds of non-questions, including: repeat questions, inquiries about previous transactions, questions that are unclear or out of scope, and out-and-out spam. These all need to be dealt with, one way or another, before the real questions can be answered. Some of this filtering can be done automatically, but there are always cases which require human judgment.
At the same time, there are steps that can be taken to improve the efficiency of the answering process. Stock answers can be sent to frequently asked questions. Questions can be categorized in various ways: by subject, by patron affiliation, by estimated degree of difficulty, and so on. Questions can be assigned to answerers, based on their capacity or known areas of expertise. If answerers are instead expected to select from the set of active questions, then the questions can be briefly summarized to aid in this process.
Advanced digital reference services, including the ones described below, assign one or more individuals specifically to carry out these tasks. These individuals are able to make judgments about the content of questions, including whether a given question falls within the scope of the service, and are able to identify repeat questions, even if worded slightly differently. They know how to identify questions that would be best referred to other services or that can be fulfilled by stock answers, asking patrons for clarification if necessary. Finally, they are able to categorize questions, which increases the efficiency of the answerers, and is also useful for later generation of statistical reports. Depending upon the policy of the service, the filterers can assign questions to individual answerers. All of these measures free the answerers to do what they do best: answer the real questions.
We, along with most of the other services described here, have found that the answerers perform best if they are not distracted by the operations of filtering. In addition, filtering often involves subtle questions of policy, thus requiring a higher level of judgment than answering. By separating out these functions, the staff members who are more experienced can carry out this role while those with less familiarity can play the role that requires fewer judgment calls.
At base, the role of the answerer is the customary role of the reference librarian: assisting patrons with their information needs. This is easily the most time-consuming job in digital reference, and thus the core of any service. As with the other roles, there are substantial differences between working in a face-to-face setting and working in a digital environment. On the positive side, answerers are spared the weary task of answering "Where is the bathroom?"for the thousandth time. On the other hand, a never-ending flow of challenging questions makes the job much more intense than working the reference desk of your local library.
Without patrons waiting impatiently in front of the desk, it becomes important to specify standards for workflow. These include: how often should the in-box be checked; how much time should be spent, on average, in answering each question; how quickly should a response be returned. In theory, answerers can work anywhere they can get an Internet connection. This could range from a public reference desk (while not otherwise occupied), to a private office, to their own home. Different answerers may work best on different types of questions. For example, it may make sense to designate some answerers as subject specialists and others as generalists. Some may work best on locating sources while others excel at looking up specific answers. The possibilities are endless.
Once the answers are generated, the digital environment makes it possible to check and archive them. This is not a necessary part of a digital reference service, but can greatly enhance the quality of the service. Guidelines for the format and content of replies can be put in place and checked, and in appropriate contexts supervisors are able to review answers for accuracy and completeness. This can be very useful, for example, in training new staff members. Finally, questions and answers can be archived in a database for later retrieval either by the staff or by the public at large. As noted above, this can be valuable, but it also raises concerns about privacy and intellectual property rights.
Any system which employs more than one or two people will inevitably require a certain amount of attention to smoothing the flow of work. This is the role of the administrator. While not as labor-intensive as the role of answerer, it is equally crucial. The tasks performed by administrators clear the way for the answerers and filterers to do their jobs properly. Equal parts problem-solver, policy-enforcer, and babysitter, administrators are the watchful eyes that keep service consistent and running smoothly on a daily basis.
In some cases, administrators may take on the role of making sure that every question is answered promptly. They may also be responsible for clean-up duties such as double-checking answers before or after sending them to patrons, transferring answered questions to archives, and collecting statistics about the operation of the service. There may also be low-level technical tasks that administrators could assume such as creating accounts for answerers, or monitoring the software used to operate the service in case glitches prevent patrons from submitting questions or answerers from accessing the system. The precise tasks that need to be done depend in large measure on the particular software and procedures used.
The final role necessary for the successful operation of a digital reference service is that of overseeing the "big picture."This is the role of the coordinators, who are responsible for defining and implementing the policies and procedures that make possible the operation of the service. This role may involve tasks such as: choosing software; setting down procedures and getting feedback about them from the rest of the staff; training new staff members; and making personnel decisions. This is in many ways similar to the role played by the coordinators or directors of any other reference service. The main difference that we have found in the digital world is that it is very important to have in place policies and procedures that are clear and well-understood. Whereas a librarian behind a typical reference desk can always fall back on instinct augmented by lessons from library school, the digital world presents many new issues that can not be handled informally in a satisfactory manner. Therefore, it is all the more important to be clear about who has primary responsibility for setting down the policies and ensuring that they are adhered to.
In order to make these decisions, coordinators need to stay up-to-date on the latest work on theory and practice in the digital reference community. A few years ago, this community was a relatively small one. With recent studies indicating that 45 percent of academic libraries and 12.5 percent of the public libraries in the United States are offering digital reference services, there is an increasing body of literature and opportunities for education. 1 Lastly, in order to keep their service viable in the long run, the coordinators must be able to articulate a vision for their service and market it successfully to management, staff, and patrons.
To illustrate how this model can be used in practice, we apply it to some real-world examples. We can characterize the services described below by detailing how each of the standard roles fits in to the particular reference process used by each service.
Case One: The Internet Public Library
This service is the one with which both authors are affiliated. 2 Since the inception of its online reference service in 1995, the Internet Public Library (IPL) has received more than forty-five thousand questions (see table 1 for a year-by-year breakdown). The process we use to answer questions is shaped by two factors: first, that we accept questions from the general public on nearly any topic; and second, that many of these questions are answered by library students and librarian volunteers. We have evolved a procedure that is aimed at making the best use of a small but highly trained paid staff by entrusting them with most of the difficult decisions and judgment calls. The much larger group of students and volunteers carry out the more straightforward - although still challenging - job of answering questions.
Table 1. Questions Received by the Internet Public Library, August 1995-March 2001
The patrons of the IPL come from all over the world, and range from schoolchildren to executives. In order to make the best of our limited communication channel, we have designed an extensive Web form that asks the patrons for information about themselves and their question. We have found over the years that this helps us to provide the patrons with the best possible answers. However, there is a negative side to this issue as well. In order to generate an archive of answered questions and at the same time preserve the privacy of our patrons, we must strip out all personal information before the questions are archived. Much of this can be done automatically, but some patrons insist on including personal information in the text of their question, and we have been unable to identify any way to extract this other than manual editing.
The roles of filterer and administrator are those which require the most training and skill. Accordingly, we have tended to combine these jobs together. Our most experienced staff are the ones who handle a question at both the beginning and end of its life cycle. We have developed written policies and procedures, including guidelines for making the necessary decisions regarding which questions to accept, reject, or refer. At the same time, our best filterers have been those who have strong customer service skills, along with the ability to make sensible decisions about how to handle questions which fall into gray areas. We affectionately refer to our filterers as "muckers,"since the job of working through a very full in-box can be compared to wading for hours through muck. In order to ease the monotony, they periodically switch to the somewhat easier task of reviewing and archiving answers, and forwarding thank-you notes to the answerers. One person is put in charge of the service for each twenty-four-hour period; they have the responsibility of making sure that every question is answered in a timely manner, and that we do not accept more questions than we have the resources available to answer.
The work of answering the questions is performed by a large, diverse, and geographically widespread group of people. Because they are for the most part doing this in their spare time, we cannot rely on specific levels of performance. At the same time, our domain of knowledge is so broad that we cannot assign questions based on prior knowledge of subject expertise. Rather, we post the available questions and allow the answerers to choose the ones they wish to work on. Those answerers who have specific goals (such as students whose coursework involves answering questions for us) can be tracked over time to determine whether those goals have been met. We have developed written guidelines for the answering process, but as a precaution each answer is reviewed by an administrator, and a correction or follow-up sent if necessary.
The coordination of a service such as ours is definitely a full-time job. The duties include recruiting, training, and supervising the filterers, administrators, and answerers who keep the service operating on a daily basis; updating the written policies and procedures in response to new situations; generating and interpreting statistical reports; and taking a turn at the other three jobs in order to keep abreast of current issues as they come up. The coordinator is also responsible for handling problem patrons, making sure that extremely difficult questions are dealt with, and finding ways to control question volume.
Case Two: Saskatchewan Libraries: Ask Us!
Our familiarity with the procedures used by the Saskatchewan Libraries: Ask Us! online reference service is based on our role as consultants in helping to set up this service. 3 It has been operating since January 2000, and has answered approximately fifty to seventy-five questions per month. 4 Because it is operated by a geographically focused institution, the character of the service is much different than that of the IPL.
The patrons of this service fall into two classes: patrons of the Saskatchewan library system who have general questions, and people from outside the province who have questions about Saskatchewan. In order to most efficiently serve the former group, the patrons are asked to provide their postal code. The province is divided into ten library regions, and this allows each question to be directed to the patron's regional main library. Many of the questions received by this service concern the availability of books and other services at the patron's local library, and they are thus assured that the question will be looked at first by a librarian from the same region. Questions from outside Saskatchewan are put into a separate pool which is checked by all of the librarians.
Because of the relatively small volume of questions and the fact that all of the staff are trained librarians, the roles of filterer, answerer , and administrator are combined. The staff of each regional library is free to distribute responsibility in any way they choose, but the typical procedure is for a single librarian to handle a question from start to finish. Occasionally the original librarian is unable to answer the question, and will refer it to another regional library or to a shared "stumpers" question pool. The coordinator of the service works for the provincial library, and works with a liaison from each regional library to set common policies.
Case Three: Virtual Reference Desk Network
Sponsored by the U.S. Department of Education, the Virtual Reference Desk (VRD) project works to support digital reference services of all varieties (library-based and expert-based) in their efforts to provide human-mediated, Internet-based information services to specific communities or the general public. 5 One of the services that this project provides is the VRD Network, to which participating digital reference services can submit out-of-scope and overflow questions. 6 These questions are then routed to another participating service, or are answered "in-house" by VRD staff members or volunteer information specialists (mainly librarians and library school students.) Since it began operating in January 2000, the network has received 4,231 questions, of which 75 percent were answered in-house. 7 This kind of collaboration among digital reference services makes a useful case study, particularly for other organizations who may be contemplating starting similar cooperative efforts to exchange or re distribute questions.
The majority of the patrons who are served by the network are referrals from the various participating services. A small number of patrons write directly to the network, often because they had previously had a question answered in-house by a VRD information specialist. As such, the patrons are a diverse group, and little information may be known about them. It is up to each referring service to determine how much information to collect about the patron, and some collect more than others. Privacy issues are important in collaborative ventures such as these. Participating services, for example, must be aware of their responsibility to alert patrons of the possibility that their question might be forwarded to another service. The network also has to be protective of patron privacy when adding questions that are answered in-house to their own archive of questions and answers.
A relatively small operation, the VRD Network dedicates only one full-time staff member and one part-time staff member to the tasks involved in operating the service. The roles of filterer and administrator are combined, and are handled on an alternating monthly basis by the two staff members. The filtering tasks carried out each day include sorting through the incoming questions, selecting which questions to refer to participating services or to an information specialist, and notifying patrons of the status of their question. Some of the daily administrative tasks include spot-checking in-house answers, and editing and archiving answered questions. Having the more highly trained staff handle these more complicated tasks allows the volunteer information specialists to focus on the role of answerer. Answerers are first trained to ensure that their answers comply with service policies, and are then placed on duty on an every-other-month basis. Once an answerer has completed a question, they are sent a new question, at the rate of about two to three questions per week per volunteer. VRD staff members assist in answering questions, too, as their time permits. Services which submit questions to the network are required to answer any questions which get routed to them, but there are no specific policies as to the type of answer they must provide, nor a specific time limit.
The role of coordinator is handled by the network's full-time staff member, Blythe Bennett, outside of her filtering, administering, and answering duties. Coordinating a collaborative effort on this scale is not a simple task. The service needs to recruit and train volunteers to serve as information specialists, develop and maintain policies and procedures for answering and routing questions, and nurture relationships with the participating services and potential participants.
Uses of the Model
We hope that the model described here will facilitate further research by providing a framework and terminology for discussion about the digital reference process. With further work, it may be possible to develop a comprehensive process model for digital reference.
This model can be used to facilitate the practice of digital reference in a number of different ways. Perhaps its most important use is in the development of written policies and procedures. We have found that it is helpful to organize these by role. This helps to clarify the interactions between roles, and to help each participant in the reference process understand what is expected of them. At the same time, if the written guidelines cover the major aspects of each role, they can be assumed to be reasonably complete.
Another area in which this model can be of use is in the choice of software for coordinating the operation of a digital reference service. This choice is critical to the success of such a service, in that the software must be able to facilitate each person's work in carrying out his or her role in the process. One good way to evaluate such software is thus to examine it from the point of view of each role. The question to ask is: does the software provide the functionality necessary to enable each person to carry out the tasks assigned to their role, in accordance with the procedures that have been established? In addition, the software must provide efficient channels for communication between the participants. This includes not only communication between the patron and the librarian, but also between the librarians in their various roles.
Finally, the model can be useful in evaluating the performance of a service and identifying bottlenecks. If accurate statistics are kept regarding the actions taken under each role in the process, one can determine where improvements in the procedures associated with each role might enhance the throughput of the entire service.
We have described a model that delineates five roles involved in the digital reference process and have described some of the aspects of each role. Compared to traditional desk reference, the digital environment presents novel challenges. However, it also allows for increased efficiency by letting staff members focus on one aspect of the job at a time. We found during the course of developing our own digital reference service that the language used to discuss traditional desk reference was simply not adequate to describe this separation of roles in a new and more complex domain. We hope that this new model will provide a basis for further discussion and research about the process of digital reference and will also provide a framework upon which decisions about digital reference practice can be made.
Our experience with digital reference shows that running a successful service depends upon having clearly defined policies and procedures that are well understood by all the participants. By framing these policies and procedures in light of these roles and the interactions between roles, one can ensure that all necessary aspects of the service have been covered and that everyone knows which roles they have been assigned to play. As is the case in so many domains, having an appropriate terminology on which to base discussion and decision contributes greatly to the effectiveness of the results.
References and Notes
1. J. Janes, D. Carter, and P. Memmott, "Digital Reference Services in Academic Libraries," Reference & User Services Quarterly 39, no. 2 (winter 1999): 145-150; J. Janes, "Current Research in Digital Reference," VRD 2000 Conference Proceedings. Accessed July 9, 20011, www.vrd.org/conferences/VRD2000/ proceedings/janes-intro.shtml.
2. The Internet Public Library's Ask a Question Web site. Accessed Apr. 23, 2001, www.ipl.org/ref/QUE.
3. The Saskatchewan Libraries' Ask Us! Web site. Accessed Apr. 23, 2001, www.lib.sk.ca/askus.
4. M. Bennett, "Saskatchewan Libraries: Ask Us! Pilot Project Evaluation Report and Recommendations,"Accessed Ap. 23, 2001, www.lib.sk.ca/staff/vitref/askusreport.html.
5. The Virtual Reference Desk. Accessed Apr. 23, 2001, www.vrd.org.
6. More information about the VRD Network can be found at www.vrd.org/network.shtml (accessed Apr. 23, 2001).
7. Statistics and other information about the behind-the-scenes operation of the VRD Network were provided by Blythe Bennett, the VRD Q&A Coordinator, via personal e-mail to Patricia Memmott, April 22, 2001.