Assessment

How do you recognize success—or failure? Are your patrons repeat users or do they quit after a single bad experience? Are you providing virtual reference in ways users find helpful, or is there a disconnect? Are users aware of the services you offer in the first place?

Evaluating transcripts, examining patterns of use, or soliciting feedback from users can help answer these questions. Additionally, many libraries have well established best practices that can be applied. A variety of tools, approaches, and frameworks for assessment are available to measure patron satisfaction, evaluate service quality, define training needs, evaluate service visibility, and more.

Framework/Measures

Kern, M. K. "Looking at the Bigger Picture: An Integrated Approach to Evaluation of Chat Reference Services." Reference Librarian 46, no. 95 (2006): 99-112. doi:10.1300/J120v46n95_07

Luo, Lili. “Chat Reference Evaluation: a Framework of Perspectives and Measures.”Reference Services Review 36, no. 1 (2008): 71–85. doi:10.1108/00907320810852041.

Novotny, Eric. Assessing reference and user services in a digital age. New York: Haworth Information Press, 2012.

Wan, Gang (Gary), et al. Key Issues Surrounding Virtual Chat Reference Model: A Case Study. 37, no. 1 (2009): 73-82. doi:10.1108/00907320910937299

Service Quality/Customer Service

Kwon, Nahyun, and Vicki L. Gregory. "The Effects of Librarians' Behavioral Performance on User Satisfaction in Chat Reference Services." Reference & User Services Quarterly 47, no.2 (2007): 137-48. doi: 10.5860/rusq.47n2

Maximek, Sarah, Eric Rushton, and Elizabeth Brown. “Coding into the Great Unknown: Analyzing Instant Messaging Session Transcripts to Identify User Behaviors and Measure Quality of Service.” College & Research Libraries 71, no. 4 (2010): 361–373.

Pomerantz, Jeffrey, Lili Luo, and Charles McClure. “Peer Review of Chat Reference Transcripts: Approaches and Strategies.” Library & Information Science Research 28, no. 1 (2006): 24–48. doi:10.1016/j.lisr.2005.11.004

Turel, Ofir, Catherine E. Connelly, Glenda M. Fisk. “Service with an e-smile: Employee authenticity and customer use of web-based support services.” Information & Management 50, no. 1-2 (2013): 98-104. doi:10.1016/j.im.2013.02.004

Weismantel, Arlene and Christine Tobias. "Exploring Service Quality in Cooperative Reference." Video, 6:48. Posted January 31, 2012. http://www.youtube.com/watch?v=rrdbOe2du10&context=C4d3d4dcADvjVQa1PpcFO35LBKEqBI_45Fmu5_KF1Ql3M_KYVhits

Top of Page

Service Process/Usage Statistics

Graybill, Jolie Ogg, and Tracy Bicknell-Holmes. "Location, Location, Location: The Impact of IM Widget Placement." College & Undergraduate Libraries 20, no.1 (2013): 72-86. doi:10.1080/10691316.2013.761081

Houlson, Van, Kate McCready, and Carla Steinberg Pfahl. "A Window into our Patron's Needs: Analyzing Data from Chat Transcripts." Internet Reference Services Quarterly 11, no.4 (2006): 19-39. doi:10.1300/J136v11n04_02

Naylor, Sharon, Bruce Stoffel, and Der Laan Van. "Why Isn't our Chat Reference used More?: Finding of Focus Group Discussions with Undergraduate Students." Reference & User Services Quarterly 47, no. 4  (2008): 342-54. doi:10.5860/rusq.47n4

Nolen, David, Amanda Clay Powers, Li Zhang, and Yue Xu . “Moving Beyond Assumptions: The Use of Virtual Reference Data in an Academic Library.” Portal: Libraries and the Academy 12, no. 1 (2012): 23–40. doi:10.1353/pla.2012.0006

Powers, Amanda Clay, Julie Shedd, and Clay Hill. "The Role of Virtual Reference in Library Web Site Design: A Qualitative Source for Usage Data." Journal of Web Librarianship 5, no.2 (2011): 96-113. doi: 10.1080/19322909.2011.573279

Top of Page

Guidelines and Competencies

Maness, Jack M., Sarah Naper, and Jayati Chaudhuri. "The Good, the Bad, but mostly the Ugly: Adherence to RUSA Guidelines during Encounters with Inappropriate Behavior Online." Reference & User Services Quarterly 49.2 (2009): 151-62. doi:10.5860/rusq.49n2

Pittman, Kim and Gardner, Gabriel. “Ask Us! Creating, Assessing, and Improving Chat Reference Services in Your Library, 2012 . (presentation, Minnesota Library Association's Academic and Research Libraries Division (ARLD) Day, Chaska, MI, 27 April 2012)

RUSA Task Force on Professional Competencies. “Marketing/Awareness/Informing - Assessment.” Professional Competencies for Reference and User Services Librarians, January 26, 2003. http://www.ala.org/rusa/resources/guidelines/professional

Thomsett-Scott, Beth. Implementing virtual reference services. Chicago: American Library Association, 2013.

Top of Page