Prepared by the ACRLÂ Standards and Accreditation Committee
1997
Adams, Roy, et al. Decision Support Systems and Performance Assessment in Academic Libraries. New Providence, NJ: Bowker-Saur, 1995.
This rather interesting report is a sponsored research project of a British Library, Leicester Polytechnic, done in the late 1980s. Its purpose was to show in step by step detail how to purchase an off the shelf decision support software package, a detailed criterion for selection was given, and then install it and make it a contributing member of your systems area. This report is long winded in some areas, but overall it is informative and thought provoking. One area in the conclusion touches on the importance of administration commitment such system into a library.
American Library Association. Library Administration and Management Association. Library Research Roundtable. Reference and Adult Services Division. Library Effectiveness: A State of the Art: Papers from a 1980 ALA Preconference. Chicago: American Library Association, 1980.
Consists of a collection of 24 papers from a 1980 ALA Preconference. Topics include Managerial Rating, Collection Size, Library Surveys, and User Oriented Approach to Assessment. Six papers are specific to public library assessment and five address reference services.
Aversa, Elizabeth. "Organizational Effectiveness in Libraries: A Review and Some Suggestions." Drexel Library Quarterly 17, no. 2 (1981), 27-45.
Aversa addresses some of the criteria that have been used in measuring library effectiveness. She evaluates the basic models and recommends a systems approach based on efficiency rather than effectiveness.
Boles, Janet. "Statistical Software for Libraries: a Review of OUTPUTM." Computers in Libraries 9 (June 1989), 28-29.
Boles's short, but helpful article, is simply a review of the statistical software mentioned in the ALA's Output Measures for Public Libraries, with the intent of helping librarians who are engaged in this effort. Boles writes in simple terms and librarians can make the best use of this paper when in front of a computer running this program. It is just what it claims to be, a review of software.
Boyett, Joseph and Henry P. Conn. "Developing White-Collar Performance Measures." National Productivity Review 7 (summer 1998), 209-218.
Few measures are available to gauge the performance of white collar managers, who now represent 55+ percent of the U.S. work force and may represent 90+ percent of the work force by the year 2000. Boyett and Conn reinforce what is already known, viz., that there are problems in trying to measure the performance of these workers. Nonetheless, they offer some suggestions if future attempts are made to measure the performance of these workers. They suggest some approaches that include measuring results, not activities; the use of group or team based measures; and, the use of a family of indicators. With concepts taken from years in the business world they offer some thought-provoking ideas that center around the involvement of these white collar workers themselves in any search for measurement.
Braunstein, Yale M. "Library Funding and Economics." in: Rethinking the Library in the Information Age, V. II. Washington: U.S. Office of Educational Research and Improvement, Office of Library Programs, 1988, 201-215.
Braunstein believes that libraries, as non-profit service organizations, are difficult to measure in terms of productivity. In order to measure productivity better, output measures such as circulation and the effect of the library on its parent organization should be explored. Libraries must be measured on both their effectiveness and their efficiency. In studying the area of performance measurement, it should be noted that the more direct the focus is on library operations and on its costs the better the chances for success.
Broom, Cheryle A. "Performance Based Government Models: Building A Track Record." Public Budgeting and Finance 15, no. 4 (winter 1995), 3-17.
The first part of this article offers a summary of strategic planning, performance-based measures, and result-oriented budgeting developed by five states. The process and the product, in the form of a list of important elements of performance/outcome based budgeting, is provided. The author also identifies critical success factors and lessons learned from statewide efforts and devotes a few paragraphs to the discussion of each factor. Even though the overall approaches that the five states pursued in establishing a performance-based/outcome-oriented budgeting process were very similar, the manner of implementation varied from state to state, providing an interesting comparison.
Calvert, Phillip James. "Library Effectiveness: the Search for a Social Context." Journal of Library and Information Science 26 (March 1994), 15-21.
Calvert reports on a survey of public library effectiveness in New Zealand. Libraries are viewed as social agencies which must be responsive to the needs and wishes of various constituencies. The author suggests that in the process of analyzing library effectiveness, it is necessary to determine not only what was done but also which of these tasks the library was supposed to do. Calvert enumerates and discusses many service indicators which have been grouped into broader service categories and subsequently were used in measuring library effectiveness.
Carbone, Pierre. "Survey of the Development of Library Performance Measures in France." I NSPEL 27, no. 3 (1993),196-198.
Carbone begins with an overview of the challenges and changes which public and academic libraries have undergone in the last twenty years in France. These developments resulted in greater complementarity between different libraries and created the need for libraries to clearly define their role, mission and place in local and national networks. In the course of library evaluation, academic libraries were among the first to pursue this area of investigation. The author identifies several surveys and conferences on library evaluation.
Clark, Tobin de Leon and Elmer U. Clawson. "Output Measures for Evaluating the Performance of Community College Learning Resources Programs: a California Case Study." Advances in Librarianship 17 (1993), 175-202.
The study of learning resources programs in California was undertaken to develop output measures and to produce a data-collection manual for two year colleges. In addition, through this survey the authors attempted to address whether there is a need for the adoption of statewide standards. The authors provide detailed discussions of events leading to the development of these output measures as well as review of statewide survey results.
Cronin, Mary J. Performance Measurement for Public Services in Academic and Research libraries. Washington, DC: Association of Research Libraries, 1985.
Cronin looks at input and output measures in public service operations and addresses specific functions including document delivery, collection development, provision of information, and library instruction. She concludes with a look at the future that is surprisingly still quite relevant.
Cross, Kelvin F. "For Good Measure." CMA - The Management Accounting Magazine 66 no 3. (April 1992), 20+
Setting and attaining a few key performance measures within various units in an organization is essential to its effective and efficient operation. It is important to define and subsequently manage critical performance indicators. The four key performance measures include an increase in quality and delivery, and a reduction in production time and waste. Measurement of performance improvement must not only be continuous but also its rate of progress must keep ahead of the competition.
Cummins, Thompson Randolph. "Demand Analysis: Inputs, Outputs, Outcomes, and Productivity." Public Libraries 27 (spring 1988), 10-13.
Cummins describes productivity analysis as a way for public libraries to work smarter and to make more effective use of resources already committed and in place. The real challenge of productivity analysis involves balancing the need to change what is inappropriate, irrelevant, or inefficient with the equally important need to strengthen what is worthwhile and sound.
De Prospo, Ernest R., et al. Performance Measures for Public Libraries. Chicago: Public Library Association, 1973.
De Prospo discusses the progress on the Measurement of Effectiveness of Public Library Service Study which grew out of an accreditation project by the Public Library Association. It describes new ways of gathering library statistics and applying them to decision making. A bibliography is included.
Du Mont, Rosemary Ruhig. "A Conceptual Basis for Library Effectiveness." College & Research Libraries 41, no. 2 (1980), 103-111.
Du Mont proposes a systems-based model that emphasizes the continuous process of being effective. The model is responsive to the environment and accommodates change.
Edwards, James B. The Use of Performance Measures. Montvale, NJ: National Association of Accountants, 1986.
Edwards focuses on performance measures used by management accountants for internal purposes in many different industries (not libraries in particular). Examples of measures include output variances, overhead variances, and profitability.
Evans, Edward, et. al. "Review of Criteria Used to Measure Library Effectiveness." Bulletin of the Medical Library Association 60, no. 1 (1972), 102-110.
This article contains the results of a study based on a literature review of library effectiveness. While the focus is medical libraries, the information is relevant to other libraries, but the information is somewhat dated.
French, Beverlee A. "Library Performance Measures." College & Research Libraries News no. 2 (February 1987), 72+
French describes the work of an ACRL committee charged to oversee the publication of an initial set of performance measures for academic libraries. A bibliography is included.
Goodall, Deborah. "Performance Measurement: A Historical Perspective." Journal of Librarianship 20, no. 2 (1988), 128-144.
Goodall has compiled a comprehensive historical review and assessment of performance measurement literature since the 1960s. The author concludes that future work is required to develop standard output measures in assessing the effectiveness of library services.
Hernon, Peter. "Utility Measures, Not Performance Measures for Library Reference Service?" RQ 27, no. 4 (1987), 449-459.
Hernon describes his in-depth examination of the use of performance measures for reference service, paying particular attention to those service areas that require attention and improvement. The author encourages managers to renew their commitment to those priorities that meet the information needs of library users.
Kantor, Paul B. "The Library as an Information Utility in the University Context: Evolution and Measurement of Service." Journal of the American Society for Information Science 27, no. 2 (1976), 100-112.
Kantor has written a highly technical examination of the problem of the objective measurement of library service performance based on the multi-dimensional and aggregate factor of "Total Contact Time per Potential User." The author provides a thorough analysis of the proper versus improper modification of goals and performance measures for library planning and decision making systems.
Kaplan, Robert S. and David P. Norton. "The Balanced Scorecard: Measures that Drive Performance." Harvard Business Review 70, no. 1 (January-February 1992), 71-79.
As part of a year-long research project with 12 companies at the leading edge of performance measurement, a "balanced scorecard" was developed to give managers a set of measures providing a fast but comprehensive view of their business. The scorecard provides answers to four basic questions: how do our customers see us? at what must we excel? can we continue to improve and create value? how do we look to our shareholders?
Kelley, Patricia M. "Performance Measures: A Tool for Planning Resource Allocations." Journal of Library Administration 14, no. 12 (spring 1991), 21-36.
Kelley introduces the reader to the concept of performance measures and data that indicate to managers how efficiently or satisfactorily the library serves it clientele. How performance measures differ from current data collection practice, why performance measures are needed, and how measurement can assist a library's planning process are emphasized.
Lancaster, F. W. The Measurement and Evaluation of Library Services. Washington: Information Resources Press, 1977.
In his extensive and informative examination, Lancaster discusses techniques that can be used to evaluate library public and technical services through the use of objective procedures. The author places a particular emphasis on evaluation methodology in determining whether a library meets the immediate tangible needs of its users.
Linke, R.D. "Some Factors for Application of Performance Indicators in Higher Education." Higher Education Management 4 (1992), 194-208.
Linke was commissioned by the Australian Department of Employment, Education, and Training to chair a study of performance indicators in higher education. Following actions by the Australian government to establish a funding system in higher education which would take into account institutional performance, Linke attempts to describe certain principles of institutional performance appraisal. The principles outlined are intended to provide a practical framework for developing a more comprehensive and effective indicator system.
Maguire, Carmel and Patricia Willard. "Performance Measures for Libraries: Statistical, Organizational, and Cosmetic." Australian Academic and Research Libraries 20 (December 1989), 262-273.
Building upon the theoretical models proposed by Orr and Buckland, the authors examine problems of measurement.
Maxstadt, John M. "A New Approach to Reference Statistics at Louisiana State University." College & Research Libraries News 49, no. 2 (February 1988), 85-86.
Collecting reference statistics is considered a waste of time by many because it is time-consuming, and statistical systems are designed without a sound scientific basis. In 1986/87, the LSU library developed a new approach which would collect statistics by both question type and patron type. To free the reference desk professional staff to concentrate on the information needs of patrons, they determined that a random statistical sample of 60 hours per year would provide sufficient data with an acceptable confidence level. New statistics gathering sheets were designed, and using graduate assistants, the new approach yielded 53 separate data elements. In general, the new sampling approach to reference desk statistics has been highly successful and is recommended for emulation by others.
McClure, Charles R., et al. "Output Measures: Myths, Realities, and Prospects." Public Libraries 25 (summer 1986), 49-52.
The authors review the effectiveness and value of the original Output Measures for Public Libraries (1982) and suggest areas of improvement for revised standards. They assert that problems with the publication include: (1) confusing utility with precision of the measures; (2) assessing the value of the measures for unintended objectives; and (3) invalidly attempting to generalize measures from specific situations and contexts. Suggestions for improving the new output manual include: (1) providing additional information about interpretation and meaning of the various measures; (2) acknowledging the trade-off between quality of the data and the level of effort required to produce them; (3) suggesting situations in which use of a particular measures is or is not appropriate; (4) developing data collection procedures that attempt to minimize measurement imprecision; (5) integrating measurement better with overall library planning and decision making; (6) including a broader range of measures; and (7) informing users of the level of confidence that can be associated with data resulting from a particular sample size.
McClure, Charles R. "A View from the Trenches: Costing and Performance Measures for Academic Library Public Services." College & Research Libraries 47 (July 1986), 323-336.
Increased concern about the efficiency and effectiveness with which academic libraries operate has spurred the consideration of method-ologies (1) to cost selected services/operations, and (2) to develop performance measures for such services/operations. McClure reports the results of a pilot study to identify the issues and concerns of public service middle managers on costing and the use of performance measures. The findings suggest that participants have little faith in the usefulness of producing cost data and using performance measures. The study participants believe: (1) the availability of such data rarely has an impact on decision making; (2) in-house data frequently lack reliability and validity; and (3) they are too understaffed to take time away from the provision of services to identify, collect, and analyze such data.
Measuring Quality: International Guidelines for Performance Measurement in Academic Libraries. (IFLA Publication, v. 76) New Providence, NJ: Bowker-Saur, 1996.
This monograph is the product of an IFLA working group which evaluated the existing literature that is designed to measure academic library effectiveness not efficiency, and is concentrated on user-oriented indicators. Each included indicator is designed to assess either the quality of the library's overall performance, or the quality of a specific service or activity. The publication includes a comprehensive bibliography (37 pages) of literature dealing with performance measurement. Also included are English, French, German, Russian, and Spanish glossaries.
There is a general introduction on the need for, and use of, effective management tools in academic libraries. The discussion covers: quality, quality management, mission & goals, stakeholder approach, performance measurement, measuring outcome, and results of performance measurement. More detailed discussions are included in the chapters, "The Measurement Process" and "Cost-effectiveness."
Seventeen performance indicators are described in detail in the following seven categories: general library use and facilities, collection quality, catalog quality, availability of documents in the collection, reference services, remote use, and user satisfaction.
Measuring the Performance of Nonprofit Organizations: the State of the Art. Sarasota, FL: The American Accounting Association, Government and Nonprofit Section's Committee on Nonprofit Entities' Performance Measures, 1989.
The American Accounting Association is a voluntary organization of persons interested in accounting education and research. The Association's Committee on Nonprofit Entities' Performance Measures produced this document, which was the result of a project focused on expanding the means by which the performance of entities within the government and nonprofit sector are assessed. Performance measures already in use were of particular interest since they could be assumed to have already met a cost-benefit criterion.
The committee focused on performance evaluation and reporting as a broad topic encompassing program evaluation. A major outcome of the project was an annotated bibliography of relevant references (76 pages).
Several resource providers, such as foundations, may require the collection of statistics on performance or of measures of service efforts. Accrediting agencies establish standards of performance which may include quantitative performance measures. The committee determined which associations and other organizations were already compiling performance information or encouraging self-assessment programs. The report is not an exhaustive source of information on performance evaluation systems in place, even at the time of publication (1989) but provides useful guidance for nonprofit organizations interested in performance measures.
The document discusses sources of performance measures, problems inherent in measurement, issues in the reporting of performance indicators, and provides a conceptual framework for performance measures. The chapter on colleges and universities gives an overview of institutions of higher education, reviews performance assessment activities (including accreditation), and discusses institutional reporting of performance indicators. Appendices provide useful information, including instruments and sample reports, as well as listings of ten outcome areas for higher education and of performance indicators for colleges & universities.
Meyer, Richard W. "Focusing Library Vision on Educational Outcomes." College & Research Libraries News no. 5 (May 1995), 335-337.
Meyer urges library leaders to articulate a library vision within the context of the overall mission and educational outcomes of the parent institution. He compares and contrasts a land-grant institution with a small liberal arts college to illustrate how institutional mission drives the library's mission.
Michalko, Jim. "Higher Education, the Production Function, and the Library." Journal of Library Administration 19, no. 4 (1993), 11-22.
"Reorganization, reallocation, and retrenchment" characterize higher education's response to increasing financial pressures. Michalko argues that the production function, an economic theory which correlates an organization's inputs with its outputs of goods and services, is now being used as a planning model in higher education. Libraries, unlike academic departments, have multiple outputs (e.g., information provision, collections, preservation of cultural heritage, etc.) which need to be reassessed. Libraries are well positioned to take advantage of technology as a determining element in their production functions.
Orr, R. H. "Measuring the Goodness of Library Services: A General Framework for Considering Quantitative Measures." Journal of Documentation 29, no. 3 (1973), 315-332.
Orr has conducted a literature review illustrating the advantages and disadvantages of quantitative measures as valuable tools to be used in the planning and management of library services. The author concludes that the effectiveness of library operations and services can best be measured through quantitative criteria.
" Performance Measures in Libraries." North Carolina Libraries 48 (fall 1990), entire issue.
This issue of North Carolina Libraries contains ten articles which address myriad issues surrounding performance measures in different types of libraries, and for different library services. A "Selective Bibliography on Library Performance Measures" concludes the issue.
Powell, Ronald R. The Relationship of Library User Studies to Performance Measures: a Review of the Literature. (University of Illinois Graduate School of Library and Information Science Occasional Paper, no. 181) Champaign-Urbana: The School, 1988.
Powell surveys the literature of user studies in relation to performance measures, examining in particular user studies themselves, data collection, and the benefits and limitations of such studies. The paper focuses on two major approaches used to determine library effectiveness and the relationship between user studies and performance measures.
Ralli, Tony. "Performance Measures for Academic Libraries." Australian Academic and Research Libraries 18 (March 1987), 1-9.
Ralli discusses issues raised by performance measurement in academic libraries: purposes of performance measures; definitions of performance measurement, effectiveness, efficiency, goals, and objectives as these concepts relate to performance measurement; definition of the objects of measurement; and areas suitable for measuring, such as collections, reference services, document delivery, and usage.
Rzasa, Philip V. and Norman R. Baker. "Measures of Effectiveness for a University Library." Journal of the American Society for Information Science 23, no. 4 (1972), 248-253.
As Rzasa and Baker state, "the purpose of this paper was to develop an adequate measure of effectiveness for university libraries to evaluate ongoing programs or in determining the impact of proposed programs." The authors believe using a cost-effectiveness analysis is the best way to select programs to be continued or to be implemented. One thought to be kept in mind, besides cost effectiveness, is how does the program in question relate to the goals of the library.
Senkevitch, Judith Jamison. "Analyzing Productivity in the Era of Accountability." Bottom Line 5 (fall 1991), 25-28.
Senkevitch critiques current methods for reporting library effectiveness, and then espouses productivity measures that measure efficiency that is, average cost (in dollars or staffing) per unit of output. She proposes there be widespread dissemination of unit cost information for library services so administrators can make comparisons and can at least highlight where trouble spots may exist.
Shapiro, Beth J. "Access and Performance Measures in Research Libraries in the 1990's." Journal of Library Administration 15, no. 3-4 (winter 1991), 49-66.
Shapiro posits that since libraries now provide information that they do not physically own, traditional (input) measures do not adequately indicate effective services. She reviews work done by members of the Association of Research Libraries, and proposes access and performance measures for libraries based on that work. Access measures are defined as the measures of the ability for users to use the mix of information resources owned by the library and information resources to which the library provides electronic access. Performance measures are defined as measuring effectiveness of library services and user satisfaction with those services. Shapiro's measures to be collected annually fall into five core areas: instruction, gate counts, electronic database access, circulation, and reference/directional transactions. Measures to be collected less frequently also fall into five areas: facilities, automation, resource sharing, services provided, and "local characteristics."
Thompson, Ronelle K. "Evaluating Library Service." Technical Services Quarterly 5, no. 4 (1988), 27-39.
Thompson describes a user survey conducted at her library to track availability of library resources. In the process, the library learned other things about the effectiveness of library programs such as instruction and collection development.
Van Heerden, Louise. "Productivity Improvement in Library and Information Services." Mousaion 12, no. 2 (1994), 56-69.
All librarians face "the era of accountability" and must now ensure that the strategic management of their enterprises focuses on improved productivity. In this article, Van Heerden, Subject Librarian, University of South Africa, concentrates on ways to increase productivity and efficiency in South African library and information services. The author also considers, within the context of strategic management, why productivity, in general, is so important; identifies factors which cause low productivity; and suggests actions to improve productivity in library and information services.
Van House, Nancy, et. al. Output Measures for Public Libraries: a Manual of Standards and Procedures. Chicago: American Library Association, 1987.
Published in 1987, this manual is intended to be used for public library development. Geared primarily toward small libraries (with at least one full-time professional staff member), medium-sized libraries, regional systems, and state libraries, the manual defines a basic set of output measures to be used to assess and evaluate public library services. Instructions are also included for collecting, analyzing, and interpreting data.
Van House, Nancy. "Output Measures in Libraries." Library Trends 38 (fall 1989), 268-279.
Van House briefly traces the history of the use of performance measures in libraries and discusses several key issues connected with the use of such measure for decision-making and problem solving. The author covers conceptual issues relating to the definition of library effectiveness, to the question of who judges effectiveness, and to the definition of information needs and uses. She also describes methodological issues pertaining to data collection methods, sampling, and statistics. Finally, she addresses management issues, describing the appropriate use and interpretation of measurement data. Librarians are encouraged by the author to draw on research in related areas when considering the problem of organizational effectiveness.
Van House, Nancy, et al. Measuring Academic Library Performance: A Practical Approach. Chicago: American Library Association, 1990.
This manual offers a set of practical output measures designed for academic and research libraries. The measures are intended primarily for internal library decision-making, performance assessment, and resource allocation. A secondary purpose is to demonstrate a library's performance to its parent organization. The measures are service-oriented and address the quality and quantity of services (excluding bibliographic instruction) delivered to users. In addition, the book includes sample reports and forms.