3.2 Reference Service and Program Effectiveness
Cost, benefit, and quality assessments of reference services provide meaningful and practical feedback for the improvement of services, staff training, and continuing education. To determine levels of service effectiveness, costs, benefits, and quality, data must be judged in light of specific library goals, objectives, missions, and standards. A variety of measures, such as quality or success analysis, unobtrusive, obtrusive or mixed observation methods, and cost and benefit analysis provide invaluable information about staff performance, skill, knowledge, and accuracy, as well as overall program effectiveness.
3.2.2 Quality Analysis - Patron Needs and Satisfaction
The perceptions and needs of patrons are important measures of the quality and impact of reference services. Surveys, combined with other measures such as numerical counts, observation, and focus groups, are commonly used to conduct comprehensive assessments of service performance and patron needs.
Traditional Reference Services
- LibQual+™ - (Association of Research Libraries, 2001). Use to measure user perceptions and expectations of library service quality. LibQUAL+ ™ surveys are used to solicit, track, understand, and act upon users' opinions of library service quality. http://www.libqual.org/
- Library Anxiety Scale (Bostick, 1993). Use to measure the construct of library anxiety in college students of all ages. The Reference Assessment Manual, 1995.
- Reference Satisfaction Survey (Van House, Weil, McClure, 1990). Use to evaluate the success of reference as determined through user opinion of the services offered. The Reference Assessment Manual, 1995.
- Survey of Public Library Users (Yocum, Stocker, 1969). Use to obtain data on patron use of services and how important they consider those same services. The Reference Assessment Manual, 1995.
Cook, Colleen, Fred Heath and Bruce Thompson. ’Zones of Tolerance’ in perceptions of library service
quality: A LibQUAL+TM study. portal: Libraries and the Academy 3 (1): 113-123, 2003.
Evaluating Reference Services: A Practical Guide. Whitlatch, Jo-Bell. American Library Association,
2000. [Chapter 4: Surveys and Questionnaires; Chapter 5:Observation; Chapter 6: Individual Interviews
and Focus Group Interviews; Chapter 7: Case Studies; Chapter 8: Data Analysis]
Identifying and Analyzing User Needs: A Complete Handbook and Ready-to-use Assessment Workbook
with Disk. Westbrook, Lynn. New York: Neal-Schuman, 2001.
Miller, Jonathan. Quick and easy reference evaluation: Gathering users' and providers' perspectives.
Reference & User Services Quarterly, 47 (3): 218-222, 2008.
Norlin, Elaina. Reference evaluation: A three-step approach- surveys, unobtrusive observations, and focus
groups. College and Research Libraries 61 (6): 546-53, 2000.
Electronic Reference Services
Arnold, Julie and Neal Kaske. Evaluating the quality of a chat service. portal: Libraries and the
Academy 5 (2): 177-193, 2005.
Carter, David and Joseph Janes. Unobtrusive data analysis of digital reference questions and
service at the Internet Public Library: An exploratory study. Library Trends 49 (2): 251-265,
Coughley, Karen. Digital reference services: how do the library-based services compare with the
expert services? Library Review 53 (1): 17-23, 2004.
Gross, Melissa and Charles McClure. Assessing quality in digital reference services: Overview
of key literature on digital reference. Information Use Management and Policy Institute, Florida
Harrington, Deborah Lynn and Xiaodong Li. Utilizing Web-based case studies for cutting-edge
information services issues: A pilot study. Reference & User Services Quarterly 41 (4): 364-379,
Luo, Lili. Chat reference evaluation: A framework of perspectives and measures. Reference
Services Review, 36 (1): 71-85, 2008.
Luo, Lili. Toward sustaining professional development: Identifying essential competencies for
chat reference service. Library & Information Science Research, 30 (4): 298-311, 2008.
Mon, Lorri and Joseph W. Janes. The thank you study: User feedback in e-mail thank you
messages. Reference & User Services Quarterly, 46 (4): 53-59, 2007.
Pomerantz, Jeffrey, Lorri Mon, and Charles R. McClure. Evaluating remote reference service: A
practical guide to problems and solutions. portal: Libraries and the Academy, 8 (1): 15-30, 2008.
Pomerantz, Jeffrey. Evaluation of online reference services. Bulletin of the American Society for
Information Science and Technology, 34 (2): 15-19, December 2007/January 2008.
Novotny, Eric. Evaluating electronic reference services: Issues, approaches and criteria. The
Reference Librarian 74: 103-120, 2001.
Radford, Marie. In Synch? evaluating chat reference transcripts. Virtual Reference Desk 5th
Annual Conference, San Antonio, Texas, November 17-18, 2003.
Ruppel, Margie and Jody Condit Fagan. Instant messaging reference: Users' evaluation of library
chat. Reference Services Review 30 (3): 183-197, 2002.
Shachaf, Pninam Shannon M. Oltmann, and Sarah M. Horowitz. Service equality in virtual
reference. Journal of the American Society for Information Science and Technology, 59 (4): 535-
550, February 15, 2008.
Shachaf, Pnina and Sarah Horowitz. Virtual reference service evaluation: Adherence to RUSA
behavioral guidelines and IFLA digital reference guidelines. Library & Information Science
Research, 30 (2): 122-137, 2007.
Stoffel, Bruce and Toni Tucker. E-mail and chat reference: assessing patron satisfaction.
Reference Services Review 32 (2), 120-140, 2004.
Ward, David. Measuring the completeness of reference transactions in online chats: Results of an
unobtrusive study. Reference & User Services Quarterly 44 (1): 46-56, 2004.
Ward, David. Using virtual reference transcripts for staff training. Reference Services Review 31
(1): 46-56, 2003.