Reference Transactions – Volume, Cost, Benefits, and Quality Selected Measurement Tools and Bibliographic References

3. 1 Reference Transactions – Volume, Cost, Benefits, and Quality
Simple tallies of reference transactions, collected daily or sampled, can be interpreted to describe patterns of use and demand for reference services. Managers commonly use transaction statistics to determine appropriate service hours and staffing. Often, volume statistics are reported to consortia to compare local patterns of use and demand to peer libraries and to calculate national norms.

Analysis of reference transactions by type, location, method received, sources used, and subject can be used for collection development, staff training/continuing education, and budget allocation. Analysis of accuracy, behavioral performance, interpersonal dynamics, and patron satisfaction during the reference interview can be used for staff training and continuing education.

Tools:

  • Variables Used to Measure Question-Answering Performance. [In: Understanding Reference Transactions: Transforming an Art into a Science. Saxton, Matthew L. and John V. Richardson, Jr. Academic Press, 2002.] ( Variables... )
  • Measures, Statistics, and Standards for Assessing Digital Reference. [In: Statistics, Measures, and Quality Standards for Assessing Digital Reference Library Services: Guidelines and Procedures. McClure, Charles, R. et. al. Information Institute of Syracuse, School of Information Studies, 2002.] ( Measures... )
  • Cost in Staffing time per Successful Question (Murfin, Bunge, 1989). The Reference Assessment Manual, 1995. Use with Wisconsin Ohio Reference Evaluation Program (WOREP) to determine the costs in staff time per successful reference question.
  • Encountering Virtual Users: A Qualitative Investigation of Interpersonal Communication in Chat Reference (Radford, Marie L., 2006). Journal of the American Society for Information Science and Technology
  • Frustration Factor and Nuisance Factor (Kantor, 1980). The Reference Assessment Manual, 1995. Use to estimate reference service accessibility (Frustration Factor) and patron time spent waiting (Nuisance Factor).
  • LAMA–NDCU Experimental Staffing Adequacy Measures (Parker, Joseph, Clark, Murfin, 1992). The Reference Assessment Manual, 1995. Used to estimate reference desk staffing adequacy through data comparison with national norms.
  • Patron Satisfaction Survey PaSS™ - (Schall, Richardson, 2002). 7-point Likert scale survey of patron satisfaction with an online reference transaction (librarian’s comprehension of question, friendliness, helpfulness, promptness, satisfaction with answer). http://www.vrtoolkit.net/PaSS.html
  • Unobtrusive Data Analysis of Digital Reference Questions and Service at the Internet Public Library: An Exploratory Study (Carter, David S., Janes, Joseph, 2002). Library Trends, 49 (2): 251-265. Study conducted to establish a methodology for the unobtrusive analysis of a digital reference service. Logs of over 3,000 questions were analyzed on the basis of questions asked (subject area, means of submission, self-selected demographic information), how those questions were handled (professional determination of subject and question nature, questions sent back to users for clarification), and answered (including time to answer) or rejected. Answers that received unsolicited thanks.
  • Wisconsin-Ohio Reference Evaluation Program (WOREP) – (Bunge, Murfin, 1983). http://worep.library.kent.edu/index.html WOREP is designed to assess the outcome of the reference transaction and to identify factors related to success or lack of success. WOREP provides diagnostic information based on input factors: collections, staff skill and knowledge, subject strengths, types of staff, types of questions; and process factors: communication effectiveness, time spent, technical problems, assistance by directing or searching with, and instruction. The WOREP report also provides both a profile of the users of a specific reference service and a comparison of the library with other libraries who have used WOREP.

References:

Hernon, Peter. Research and the use of statistics for library decision-making. Library Administration & Management 3: 176-80, Fall 1989.

Larson, Carole A. and Laura K. Dickson. Developing behavioral reference desk performance standards. RQ 33: 349-357, 1994.

Murfin, Marjorie E., and Charles A. Bunge. A Cost Effectiveness Formula for Reference Service in Academic Libraries. Washington, D.C.: Council on Library Resources, 1989.

Murfin, Marjorie E., and Gary M. Gugelchuk. Development and Testing of a Reference Transaction Assessment Instrument. College and Research Libraries 48 (4): 314-39, 1987.

Radford, Marie L. In synch? Evaluating chat reference transcripts. Virtual Reference Desk 5th Annual Digital Reference Conference, San Antonio, TX, November 17-18, 2003. http://www.vrd2003.org/proceedings/presentation.cfm?PID=231

Radford, Marie L. Encountering virtual users: A qualitative investigation of interpersonal communication in chat reference. Journal of the American Society for Information Science and Technology 57 (8): 1046-1059, June 2006.

This document is part of the Measuring and Assessing Reference Services and Resources: A Guide prepared by the RUSA RSS Evaluation of Reference and User Services Committee.