Variables Used to Measure Question - Answering Performance

3. 1 Reference Transactions – Volume, Cost, Benefits, and Quality

Variables Used to Measure Question-Answering Performance [see complete list of variables, operational definitions, literature review, and statistical formulae in: Understanding Reference Transactions: Transforming an Art into a Science. Saxton, Matthew L. and John V. Richardson, Jr. Academic Press, 2002, APPENDIX D, p.130-189]

I. Dependent Variables:

  • Accuracy: Answering Success
  • Client Satisfaction
  • Successful Probe
  • Efficiency – Accuracy/Time
  • Librarian Satisfaction
  • Cost benefit analysis – WS$/Unit of Service
  • Unique dependent variables
    • Bunge’s Composite
    • Illinois Index of Reference Performance

II. Independent Variables:

A. The Reference Environment

  • Size of collection
  • Type of library
  • Size of staff
  • Size of professional staff
  • Size of nonprofessional staff
  • Number of volunteers
  • Library expenditures
  • Library income
  • Hours of service
  • Size of service population
  • Circulation
  • Fluctuation in collection
  • Institution’s bureaucratic service orientation
  • Staff availability
  • Level of referral service
  • Arrangement of service points
  • Administrative evalutation of services
  • Use of paraprofessionals at the reference desk
  • Volume of questions

B. The Librarian

  • Experience of librarian
  • Education of librarian
  • For paraprofessionals, amount of in-service training
  • Question-answering duties
  • Librarian’s attitude toward question-answering duties
  • Duties other than question answering
  • Librarian’s service orientation
  • Librarian’s perception of the collection adequacy
  • Librarian’s perception of personal education
  • Librarian’s perception of other duties
  • Outside reading
  • Membership in associations and committees
  • Age of librarian
  • Sex of librarian

C. The Client

  • User participation in process
  • User perception of librarian’s service orientation

D. The Question

  • Subject knowledge of librarian
  • Subject knowledge of client
  • Number of sources used to answer question
  • Source of answer named
  • Type of question

E. The Dialogue

  • Business at the reference desk
  • Communication effectiveness between patron and librarian
  • Amount of time spent with user by reference librarian
  • Type of assistance provided
  • Amount of time willing to be spent by patron

This document is part of  Measuring and Assessing Reference Services and Resources: A Guide prepared by the RUSA RSS Evaluation of Reference and User Services Committee.