MOUSS Evaluation of Reference and User Services Committee

Midwinter 2002 Meeting Minutes
January 20, 2002

Present: Lisa Horowitz (chair), Lou Vyhnanek, Leslie Haas, Lanell Rabner, Judy Solberg, Susan Ware, Barb Mann, Suzanne Lorimer, Nancy Skipper, Cindy Pierard, Douglas Stehle, Christopher Coleman, Jake Carlson.

Guests: Gordan Aamot, Jeff Steely, Chris Stewart, Cheryl Holland, Rudolph Clay, Jan Kemp, Julie Yang

1. Welcome Introductions.

There were several guests and visitors interested in the issue of evaluation of reference; one or two expressed an interest in future participation on the committee.

2. Review Agenda

Lisa reviewed the agenda, explained the need to rearrange the items as these minutes reflect.

3. Old business from yesterday

There was some discussion about the RUSA/MOUSS all committee meeting the day before and the suitability of the room. Concerns were noted about interested guests being put off by the arrangement and not being able to identify what tables represented which groups. Noise was also a problem. A larger room was definitely needed.

Lisa reported that the RUSA preconference for Toronto, ALA midwinter 2003 was approved. It will be a 1 day preconference. Paula Contreras will be leading the planning of this event. ERUS committee members who volunteered to serve on this planning committee at yesterday's meeting were Mike Havener, Cindy Pierard, and Lou Vyhnanek.

4. Guidelines & NISO Standards

Lisa began by giving some background and distributing some handouts.

The National Information Standards Organization (NISO at is revising their 1995 standard on Library Statistics which did not address electronic resources or performance measures; it needs updating.

[ Added by Stehle: Details: "ANSI/NISO Z39.7 - 1995 Library Statistics. Abstract: Identifies and defines the basic data collection categories used to collect library statistical data at the national level. Intended to apply to all types of libraries, this standard will provide valid and comparable data on library services, staff, users, and collections." Available on the web at, with current revision committee charge, progress, and plan information at The definition of "reference transaction" given by NISO to gather the stat can be found on page 6 of the standard in Chapter 2, Definitions.]

A RUSA member noticed that NISO was making a change to the standards, and brought it to the attention of the RUSA board. The RUSA Board called on our committee because they know we are working on guidelines for reference statistics. RUSA would like to lobby ARL and NISO to adopt its/our standard definition of reference/reference transaction. Thus, here we are!

Lisa distributed two handouts:

1. one with the email message to Lisa asking our committee about this and giving the RUSA/ANSI, ARL, and NCES definitions of reference transactions,

2. information & sources to assist committee in making our guidelines. RUSA's definition of "reference" is hard to find on its web site, but its current definition comes from the ANSI one. Lisa suggested that all three definitions emphasized the traditional mediation and answer mode of reference and did not address the self help or instructional guidance side of current reference practices; the result being theoretically many "transactions" may not be counted using such definitions.

Open discussion on the point began. Various questions and comments were tossed out:

  • are we trying to put together a definition of "reference transactions" or a broader definition of reference itself...
  • Many times the transaction approach to quantifying reference tends to limit or ignore other types of assistance and pathways provided by reference librarians...
  • information contacts could be construed as more than face to face interaction, for instance a patron making use of an online tutorial or some type of prepared bibliographic pathfinder...
  • we could approach the matter by listing all various examples of what reference librarians do in terms of creating such information contacts for users...
  • A user centered definition is crucial over the more traditional approach of looking at reference in terms of inputs, amount of resources and staffing...
  • It was asked whether a reference transaction definition should limit itself to use of resources or should it include direction (pointing?) to "library resources"...
  • The group seemed to like the vision of a broad user centered definition that takes into account the amount of interaction users have or make of the various "guidance??" resources we make available to them...
Plan to Proceed & Assignments
  • It was agreed that we take an inductive approach listing or culling activities that make up reference services. This would also help to see that activities from the public, special, and school library reference community would be included.
  • The current sub-committee working on the guidelines will create a simple questionnaire to cull input about the activities that the reference community sees as making up reference or reference transactions (it was not clear to the secretary that this distinction had been clarified). It was thought a literature review would be helpful also.
Here is what was to be done and by whom:
  • Christopher Coleman, Judy Solberg, and Susan Ware, the subcommittee on the guidelines for reference statistics, would develop a questionnaire and send this out via e-discussion lists as this played into their current work on developing guidelines.
  • Barb Mann, Suzanne Lorimer, and Lanell Rabner would provide us and the guidelines sub-committee some literature review.
  • Lisa asked the people who took notes at the forum the previous day to announce the upcoming questionnaire in the forum summary after midwinter.
  • The entire ERUS committee should review definitions and share input on our discussion list.

5. LIBQUAL - Information Sharing

Leslie Haas and Nancy Skipper had information to share on this topic.

LibQual emerged from Texas A&M. Nancy spoke about how Cornell was using it as a model and running studies with it. LibQual focuses on what users want and their perceptions, and not so much on library service area evaluation. An interesting outcome of LibQual findings is that users do appreciate the "library as place," that an attractive atmosphere and sense of place seemed significant in how and what users thought about their libraries. Leslie stressed that LibQual focuses on broad user service perspectives and not reference services.

After some questions about the relevance of this trend to evaluation of reference and our committee, the committee decided to wait and see final results of LibQual studies. More data will come out after a year, and with the RUSA preconference in Toronto we might be able to gauge reference field interest in a specific tool playing off LibQual for reference services.... RefQual?

At 11:00 there was a 5/10 minute break - THANKS LISA!

6. RAM 2: Reference Assessment Manual

Lisa reviewed and explained that RAM stood for "Reference Assessment Manual", the first edition being done in 1995. The volume was circulated amongst the members. The volume addressed different type of assessment approaches by reviewing the literature and studies reporting results from the use and application of each assessment instrument. Lisa said the committee has always seen a practical guide with examples of application as useful.

Two authors, Matt Saxton and John Richardson, expressed interest to the committee several years ago in revamping the RAM in a second edition, to focus more on research and theory. They have now decided to use a database, which is in prototype, in favor of publishing a new book. The idea would be to provide a database in which you could select an area of service/inquiry and select methodology factors; the database would provide you with a list of articles or studies applying those methods to service evaluation. This would be like an "instrument clearinghouse" available from the RUSA website.

Lisa asked, what now? Should we proceed with a second edition of the book on our own, or tie ourselves to this web project?

  • A strong argument was made to let these author/editors follow their path and for the committee to move beyond this item which has consumed our discussion for some time. A motion was made not to pursue RAM 2, it was seconded, and passed overwhelmingly.
The question of the ERUS committee taking some version of this on was raised. Should we post examples of surveys, provide a bibliographic literature review of examples of current assessments?
  • Some felt the 1st edition of RAM still had some merit and could use some additional promotion. Another idea emerged that involved directing visitors to our website not to published surveys and studies, but to actual practitioners who were or had worked on self studies in their own practice, e.g. a who's who of ongoing practitioner efforts. It was considered difficult how to identify these, and with either a bibliographic review database or a who's who, how would the committee verify the validity and soundness of the approaches.
  • The motion was made to focus on the RUSA pre-conference and the development of reference definition guidelines. This was approved and we can return to this issue after Toronto.

7. Committee Business Discussion

Lisa addressed the various issues associated with effective and efficient accomplishment of committee business. Some discussion centered around the committee putting out a specific statement to potential new members about expected norms of participation and commitments to projects. I

t was mentioned that while it was relatively easy to put in a membership application, that the follow-up after being placed on the committee was weak and that it is a difficult process for new members to get up to speed on what is going on. Ideally subcommittees should plan on meeting and working at midwinter and annuals; some questioned how realistic this vision was.

It would be better to have minutes completed sooner - a one month deadline was mentioned as reasonable. This led to the question of appointing a secretary to take minutes. Many people supported the notion of pursuing an intern to be assigned to do this.

The current subcommittees are:

  • Guidelines for reference statistics and definition
  • Toronto Preconference
  • Literature review support to the committee as a whole on the question of reference & reference transaction definitions.

Lisa then expressed concerned over the size of the committee and needed to find out what certain committee member intentions were. People who are renewing will get back to Lisa. Lisa wondered if we should have a size limit to the committee; she proposed we limit ourselves to 12 members although this would make bringing on new members difficult for awhile given the two year terms and number of new and renewing members. There is the risk that if we take no new members to keep the size manageable, then at some point too many members could leave in which case committee roots and history are weaken. We decided that 12 was an optimal number, but the committee will move toward that size appropriately.

It was explained that subcommittee members/task force members need not be members of the Evaluation committee itself. Christopher Coleman viewed this as a valuable asset. Committee member appointments are for two years; chair or other possible positions we decide can serve 1 year terms and continue if approved.

  • Currently Jake Carlson is our website liaison to RUSA
  • Paula Contreras is the Toronto preconference chair
  • Christopher Coleman is chairing the guidelines subcommittee
  • We have no "publicity" person
  • Douglas Stehle agreed to take minutes in the future if an Intern to do this was not obtained. An intern should be sought to be our Secretary was stressed.
  • Much discussion ensued about other positions, especially having a vice-chair.

The following decisions were reached:

  • Lisa Horowitz will continue as chair to 2003 ALA National.
  • Jake Carlson will now serve as vice-chair, effective ALA Annual 2002, and take over as chair after 2003 National.
  • Intern appointment from RUSA will be sought for a secretary and minutes keeper.
  • Cindy Pierard raised the question of establishing a planning timeline for the Toronto preconference.

It was felt that the preconference subcommittee needs to report on the status of the program, planning needs, assignments, tasks, etc. by mid March, no later than end of March 2002. A deadline schedule should be in place by the end of March for this preconference. Lisa said Cathleen Bourdon should be able to tell us about registration fees for the RUSA institute/preconference; Lisa will ask her.
Lisa made mention that at ALA National, our meetings were scheduled for Saturday morning and Monday morning.

The committee adjourned at 12:05 pm.

Humbly submitted,
Douglas Stehle