Use, Usability, and Collection Assessment Selected Measurement Tools and Bibliographic References

4.0 Measuring and Assessing Reference Resources – Use, Usability, and Collection Assessment
As reference collections grow in size and format, there is an increasing need to assess their appropriateness, strengths, weaknesses, balance, and accessibility. Reference collections, both print and electronic, must be constantly assessed to determine their usefulness and relevancy to library patrons. Use and usability tests examine how often and how well visitors navigate, understand and use web sites, electronic subscription databases, free Internet resources, library subject web pages, and other web-based tools such as bibliographies, research guides, and tutorials.

Print Reference Resources

Tools:

  • In-Library Materials Use (Van House, Weil, McClure, 1990). Use to determine total number of items used in the library but not circulated. The Reference Assessment Manual, 1995.
  • Reference Collection Use Study (Arrigona, Mathews, 1988). Use to evaluate which subjects areas are most used by librarians to assist patrons and then identify any correlation with the subject areas most used by patrons to answer their own questions. The Reference Assessment Manual, 1995.
  • Strother’s Questionnaire A and B (Strother, 1975). Use to determine faculty awareness and use of reference tools The Reference Assessment Manual, 1995.

Web-based Reference Resources

Tools:

  • Formal Usability Testing – Observe as patrons use a site to perform given tasks or achieve a set of defined goals.
  • Inquiry – Use interviews, surveys, and focus groups to gather information about patron preferences and use of a particular site.
  • Inspection – Use to evaluate a site against a checklist of heuristics and design principles or simulations of typical user tasks.

References:

Battleson, Brenda, Austin Booth and Jane Weintrop. Usability testing of an academic library Web site: a case study. Journal of Academic Librarianship 27 (3): 188-198, 2001.

Kovacs, Diane K. Building a core Internet reference collection. Reference & User Services Quarterly 39 (3): 233-239, Spring 2000.

Rettig, James. Beyond cool: analog models for reviewing digital resources. Online 20 (6): 52-64, 1996.

Rubin, Jeffrey. Handbook of Usability Testing: How to Plan, Design, and Conduct Effective Tests. New York: Wiley, 1994.

Smith, Alastair. Evaluation of Information Sources. (Webliography of information evaluation resources) http://www.vuw.ac.nz/staff/alastair_smith/evaln/evaln.htm

Usability Testing of Library-Related Websites: Methods and Case Studies. Campbell, N., ed. LITA Guide #7. Chicago: LITA/American Library Association, 2001.

Collection Assessment

References:

Bucknall, Tim. Getting more from your electronic collections through studies of user behavior. Against the Grain, 17 (5): 1, 18, 20, November 2005.

Dee, Cheryl, and Maryellen Allen. A survey of the usability of digital, reference services on academic health science library web sites. Journal of Academic Librarianship, 32 (1): 69-78, January 2006.

Drane, Simon. Portals: Where we are and the road ahead. Legal Information Management, 5 (4): 219-222, Winter 2005.

Keller, Michael A. Reconstructing collection development. Against the Grain, 16 (6): 1, 16, 18, 20, December 2004/January 2005.

Puacz, Jeanne Holba. Electronic vs. print reference sources in public library collections. The Reference Librarian, no. 91/92: 39-51, 2005.

Stempter, James A., and Janice M. Jaguszewski. Usage statistics for electronic journals: An analysis of local and vendor counts. Collection Management, 28 (4): 3-22, 2003.

Strohl, Bonnie. Collection evaluation techniques: A short, selective, practical, current, annotated bibliography, 1990-1998. Chicago: Reference and User Services Association, ALA, 1999.

This document is part of Measuring and Assessing Reference Services and Resources: A Guide prepared by the RUSA RSS Evaluation of Reference and User Services Committee.