Home Publications College and Research Libraries Back Issues 2002 March March 2002 Vol 63 No 2 Weller book review

College and Research Libraries
March 2002, Vol. 63, No. 2

Book Review

Weller, Ann C. Editorial Peer Review: Its Strengths and Weaknesses. Medford, N.J.: Information Today (ASIST Monograph Series), 2001. 342p. $35.60 (members); $44.50 (nonmembers) (ISBN 1573871001). LC 00-47204.

The process of refereeing articles submitted for publication to scholarly and scientific journals is of central concern in academe and the professions. The decision to publish or not to publish is one on which sciences advance, the orderly progress of knowledge is achieved, and individual careers depend. It is a process through which, presumably, all new contributions are validated by the judgments of authors’ and researchers’ professional peers and deemed fit to join the knowledge base of the discipline.

Attention has been given to this phenomenon only during the past few decades. Prior to the 1960s, there was virtually no interest in the phenomenon, at least as a researchable topic in itself. Indeed, it is questionable that there were enough problems in the process to raise questions of its legitimacy, its pervasiveness, or its ultimate impact in the scholarly community prior to the expansion of research, of the number of research journals, and of the general level of interest in the equity of access to publishing outlets prior to the 1960s. Although it was not unknown for editors of scholarly and research journals to send manuscripts out to be evaluated by experts not immediately associated with the journals, it was not a common practice in many areas until well after World War Two and even into the 1970s in some disciplines.

The book at hand is not a piece of original research or the result of an independent investigation. The author’s purpose is much more modest. Her avowed intent has been simply “to conduct a systematic review of published studies on the editorial peer review process” from the earliest studies she could identify through her closing date of 1997. Weller presents here a highly structured approach to the organization of the reviews, beginning each chapter with an overview of the issues involved. She posits an explicit set of questions to be answered and a set of inclusion criteria for the research reports included in each section before describing those articles that address her questions and meet her criteria. Each chapter concludes with a general assessment of the research in the area treated in it and recommendations for further research. Most of these suggest more work along the same line and, for the most part, are directed toward practical ends—to improve the editorial review process. It is a practical, instrumental approach.

The array of concerns this book addresses extends much further than the simple practicality this description might suggest. Enough research has been conducted and published over the past four decades to produce a respectable showing, and she touches on every conceivable aspect of the issues involved in the process. After a general introduction to the problem, she considers studies of rejected manuscripts, the composition of editorial review boards, and the role of editors. She continues to evaluate research into the various roles of reviewers, their biases and agreements, and the use of specialized reviewers of statistical elements of research. She concludes the book with a chapter on the role of referees in the electronic environment and a final short chapter of general recommendations and observations on the editorial review process.

Through it all, she maintains a remarkably objective and descriptive tone, which, at times, is leavened with observations on the limitations and fallacies involved in the design and execution of the studies under discussion. Her assessment of the validity of utilizing fabricated manuscripts to assess the biases of referees, in particular, illustrates her excellent command of the methodologies involved.

Weller usually arrives at balanced, well-conceived conclusions, but at times, her familiarity with the topic and her commitment to the success and preservation of the refereeing process betrays her. The chapter on reviewer biases is a sound piece of work and her assessment of the research in this area is generally solid, but she does encounter some problems, probably brought about from her acceptance of the researchers’ assumptions underlying their own work. She notes, for example, that “the studies of reviewer bias must be considered in light of studies that have shown that researchers and scholars from major institutions publish more than researchers and scholars from less prominent institutions.” She cites the failure of researchers into the process to examine the phenomenon of authors resubmitting rejected manuscripts or the relationship between prolific authors and blind refereeing as possible reasons for this phenomenon. What she ignores, it seems, is the more obvious connection. It is precisely because more scholars and researchers from prominent institutions publish more than those from minor institutions that their institutions are considered prominent. If most of the reputable research in any field is published by the faculty of Northeast State College, then Northeast State College is, by definition, a prominent institution.

Her recommendation that journal editors inform readers of the level of peer review undergone by each article also strikes a dissonant chord. The notion that a subtitle would be appended to each article by the editors as a permanent identifier forming part of the bibliographic record could be appealing but is probably impractical from an editorial perspective and undesirable from an author’s view. It seems unlikely that many authors would be amenable to such a tag becoming part of their title.

To be sure, the author’s lapses of judgment are never as complete as those of some of the researchers she reviews. M. S. Kochar’s idea, published in a 1986 issue of Journal of Chronic Diseases, that the major medical journals might band together to evaluate manuscripts submitted to the collective journals represented and apportion them out to the individual journals for publication was described without irony. Weller did note, though, that the idea acquired no citations in the Web of Science at least in the following decade.

Anne Weller’s contribution to the literature of editorial peer review goes well beyond her modest attempt at reviewing the research literature. Indeed, some 25 percent of the hundreds of pieces commented on in this book are thought pieces, editorials, and policy statements rather than research. This book represents more a survey of thought about the process than research into it. It is well constructed, well written, and thorough in its coverage of the literature of editorial peer review. However, it is difficult to find an intended readership for the work. The lack of an author index makes it impossible to identify any particular piece of research or writing, and the abbreviated subject index is redundant given the clarity of the chapter divisions. It is neither a book that can be read nor one that can readily serve as a reference.

owever, it is a book that should be in any collection serving researchers and authors or graduate students aspiring to publish their research. In addition to the understanding of the process of scientific refereeing, Weller’s frequently perceptive critiques of existing research and suggestions for further research, particularly in her concluding chapter, may fuel and inspire research in the area. Weller has produced a dense, sometimes difficult, book, but one that is worth the effort.—Lee Shiflett, University of North Carolina at Greensboro.