Collection Assessment and Evaluation e-Forum

This e-forum was two days full of exploration and sharing. We had a diverse group of librarians participating, from public libraries, large public universities, as well as smaller schools, community colleges, and private schools. From this diverse audience, we learned that the roles of assessment in their libraries and jobs were also quite varied. Many were interested in learning how to use objective measures to make sound and unbiased decisions for deselection. Others, particularly at academic institutions, used assessment to ensure that their collection was balanced and fair to all subjects and departments. Finally, there was a wide range of experience levels in collection evaluation. Some had been doing it for years and others are just starting the inquiry process.

The moderators asked key questions about the collection assessment and evaluation process, including:

  • What were the participants’ experiences with collection assessment?
  • What methods and measures were most important?
  • What tools were most useful in conducting assessments?
  • How to assess a collection for curricular (versus research) needs?
  • How best to evaluate non-textual collections?

Key points that were made:

  • Constant collection assessment is important. It enables adjusting purchasing patterns quickly because of the results of assessment activities. Also, when institutions need to reduce the print collections, the process can be arduous when starting from scratch.
  • There is a lot of concern about the usage of e-resources. A librarian from a community college stated “We have built a large e-collection to serve our three locations, but do not see students moving in this direction. Our students want print yet at the same time our circulation is decreasing each semester.” There seems to still be concerns regarding how best to use e-resource usage for assessments.
  • There is a lot of interest in evaluating against your peers and using ILL statistics for making collection decisions. For instance, the University of North Texas emphasized how they use peer information to assist in the collection evaluation process.
  • Identification of core or classic titles in the catalog record can be extremely helpful when you are generating a list of what needs to be assessed. One librarian from a small liberal arts college explained how identifying these titles in the catalog record can make creating lists of what needs to be assessed and what doesn’t much easier.

We also discussed the place of citation analysis in the process and what types of citations should be included. Karen Kohn of Arcadia University had presented at Timberline (see below) and found that your collections really do affect learning outcomes. There was further discussion about what citations should be used for analysis. It was agreed that books and monographs are not easy to use. A librarian from a large university suggested that, “you should stay away from undergraduate (or even master’s level) work because, as someone mentioned, those scholars may not be using the best sources. Doctoral dissertations are a better way to go.”

Our discussion on tools had few responses, which makes me wonder if this reflects a lack of awareness or uncertainty among the e-forum contributors. However, the discussion of assessment of curricular needs was quite stimulating, with members indicating that they use course listings (mapping courses to call numbers, for instance), syllabi and assignments, and accreditation requirements (if any). A librarian from a large public university indicated that classroom materials (such as textbooks and teacher guides) “aren’t permanent and (don’t) serve a broader need.” But he, like others, indicate that they “try to cultivate relationships with faculty” to keep up with the curricular needs. A Canadian librarian looked at usage of resources from their subject-specific LibGuides as another source of data.

This discussion, combined with a side-discussion on weeding VHS, led to our next question on evaluating non-textual materials such as AV and special collections. The focus on formats and replacements was interesting in that it brought to the forefront the issue of obsolescence.

The final question was about the near-term future of collection assessment. Several people suggested that the shift in formats (from physical to electronic) would predominate collection assessment, especially for monographs. A librarian from a small private college suggested that physical collections will diminish. Bradley suggested that we (those interested in collection assessment) should consider more measures about “what our users think of our collections.” Another librarian wondered about the effect of discovery layers or systems that are more integrated with the ILS.

One exciting outcome of this forum was the development of an ALA-hosted discussion list specifically on collection assessment. This interesting development came after Lucy Lyons, from Northwestern University, shared information about her Methods & Data Bank LibGuides, as well as an informal email distribution list she maintains. After receiving dozens of requests to be added to this list, e-forum coordinator Kristin Martin jumped into action and asked the ALCTS office to create a new official discussion list and the Collection Assessment Discussion List was born.  As of Thursday, November 14, 2013, more than 113 people had signed up! We would like to thank those who have shared web sites and actual tools (see Resources section).

All in all, this was a very productive e-forum. Issues and questions were raised that will continue to be food for thought as we struggle to ensure that our collections are useful and appropriate for our users.

Resources

Beile, Penny M., David N. Boote, and Elizabeth K. Killingsworth. “A Microscope or a Mirror? A Question of Study Validity Regarding the Use of Dissertation Citation Analysis for Evaluating Research Collections.” Journal of Academic Librarianship 30, no. 5 (2004): 347–53, www.sciencedirect.com/science/article/pii/S0099133304001041

“Citation Analysis as a Tool for Collection Development and Instruction.” Presentation by Karen Kohns, Arcadia University, at the 2013 Acquisitions Institute at the Timberline Lodge, https://docs.google.com/file/d/0Bwz2gOFBtGKVUk8tOVE0czBOQXc/edit

De Groote, Sandra L., Deborah D. Blecic, and Kristin E. Martin. “Measures of Health Sciences Journal Use: A Comparison of Vendor, Link-resolver, and Local Citation Statistics.” Journal of the Medical Library Association 101, no. 2 (April 2013): 110–19, www.ncbi.nlm.nih.gov/pmc/articles/PMC3634374/

Jacobs, James A. and James R. Jacobs. "The Digital-Surrogate Seal of Approval: a Consumer-oriented Standard." D-Lib Magazine 19, nos. 3/4 (2013), http://dx.doi.org/10.1045/march2013-jacobs

Jacobs, James A. and James R. Jacobs. "Wait! Don't Digitize and Discard! A White Paper on ALA COL Discussion Issue #1a." Free Government Information (June 2013), www.freegovinfo.info/node/3961

Kellsey, Charlene and Jennifer Knievel. “Overlap between Humanities Faculty Citation and Library Monograph Collections, 2004–2009.” College & Research Libraries 73 no. 6 (2012): 569–83, http://crl.acrl.org/content/73/6/569.abstract
Kellsey and Knievel looked at twenty-eight monographs published by humanities faculty at the University of Colorado between 2004 and 2009. They found that of the monographs cited in those faculty publications, 76 percent were, in fact, owned by the university library.

Knievel, Jennifer. “Alignment of Citation Behaviors of Philosophy Graduate Students and Faculty.” Evidence Based Library and Information Practice 8, no. 3 (2013): 19–33, http://ejournals.library.ualberta.ca/index.php/EBLIP/article/view/19205
Update to “Overlap between Humanities Faculty Citation and Library Monograph Collections, 2004–2009.”

Lyons, Lucy. “Collection Assessment: NUL Data & Methods Bank,” www.library.northwestern.edu/about/library-administration/departments-of...
Excellent tool for those just getting started in collection. Gives ideas of the kinds of data that are available.

Stephens, Jane, David Hubbard, Carmelita Pickett, and Rusty Kimball. "Citation Behavior of Aerospace Engineering Faculty." Journal of Academic Librarianship. Article in Press, available 23 October 2013, www.sciencedirect.com/science/article/pii/S0099133313001262

Tools

Books for College Libraries/Resources for College Libraries
www.bowker.com/en-US/products/rcl/rcl/index.html

Bowker Book Analysis System
www.bowkersbookanalysis.com/

CHOICE Reviews Online
www.cro2.org/

CollectionHQ
www.collectionhq.com/

HathiTrust
www.hathitrust.org

NCES Academic Library Comparison
http://nces.ed.gov/surveys/libraries/compare/

WorldShare Collection Evaluation System (formerly, WorldCat Collection Assessment System)
www.oclc.org/collection-evaluation.en.html

Collection Assessment Discussion List: http://lists.ala.org/sympa/info/coll-assess

To subscribe:

  • Create an account at the ALCTS Discussion List web site (how to create account)
  • Go to the Coll-assess web site http://lists.ala.org/sympa/info/coll-assess
  • Click the "Subscribe" link in the left-hand column
  • Enter the email address where you'd like to receive the messages
  • You'll receive an email message containing a password, enter the pw. You are now subscribed.

—submitted by and e-forum moderated by Karen Harker and Pat Reese, University of North Texas