C&RL News, May 2007
Vol. 68, No. 5
by Carl Bergstrom
In 1927, two chemists at Pomona College published an article in Science, proposing that librarians could use data about citation rates to select appropriate journals for a small library collection.1 This idea has had an enormous influence on the structure and practice of scientific communication. Today, citation analysis is ubiquitous in evaluating papers, researchers, journals, departments, and fields. Not only do librarians use citation data in selecting journal subscriptions, but researchers use them when deciding where to submit their manuscripts, funding bodies in evaluating grant proposals, and tenure committees in deciding tenure cases.2 But as the influence of citation data has grown, so has criticism of its use. Much of this criticism is justified; when evaluating individual papers or researchers, there is clearly no substitute for reading and understanding the work. However, some questions—such as bibliometric analysis of the relative influence of the full contents of a journal—can only be answered by a large-scale quantitative approach. For these questions, citation data can be useful, and we should make the best possible use of it.
The scientific literature forms a network of scholarly articles, connected by citations.3 Each connection in this network—that is, each citation—reflects the assessment of an individual scholar regarding which papers are interesting and relevant to his or her work. Thus contained within the vast network of scholarly citations is the collective wisdom of hundreds of thousands of authors. My colleagues4 and I have developed a way to use the network structure of citations to improve on simple citation counts in measuring the scientific influence of academic publications. At our Web site www.eigenfactor.org, we report these measures for the nearly 8,000 publications indexed by Thompson Scientific’s Journal Citation Reports (JCR) as well as for the approximately 110,000 other journals, books, newspapers, and other reference items that are referred to by these publications.
How it works
Our approach is to rank journals much as Google ranks Web pages. While Google uses the network of hyperlinks on the Web, we use citations in the academic literature as tallied by JCR. By this approach, we aim to identity the most “influential” journals, where a journal is considered to be influential if it is cited often by other influential journals. While this might sound hopelessly circular, it is not: we can iteratively calculate the importance of each journal in the citation network by a simple mathematical algorithm.5
This iterative ranking scheme, which we call Eigenfactor, accounts for the fact that a single citation from a high-quality journal may be more valuable than multiple citations from peripheral publications. We measure the importance of a citation by the influence of the citing journal divided by the total number of citations appearing in that journal. This corrects for differences across disciplines and journals in the propensity to cite other papers. For example, a citation from a review article that has cursory references to large numbers of papers counts for less than a citation from a research article that cites only papers that are essentially related to its own argument.
Eigenfactor measures the total influence of a journal on the scholarly literature or, comparably, the total value provided by all of the articles published in that journal in a year. This seems the appropriate metric for making subscription decisions. All else equal, larger journals will have more citations and larger Eigenfactor scores and will be visited more often by researchers. If, on the other hand, one wants to estimate the importance of an article by the company it keeps, the size of the journal in which it is published is not relevant. Instead one would want to measure the average influence of articles appearing in the same journal. The measure that we use as the Article Influence for a journal is proportional to the Eigenfactor divided by the number of articles. This measure is more directly comparable to ISI’s familiar Impact Factor.
In the sidebar below, we list the top ten nonreview journals in the sciences, ranked by the Article Influence.
The Article Influence is scaled with respect to a mean of 1.00; thus Science has more than 17 times the influence of the mean journal in Thompson Scientific’s ISI database.
In addition to ranking the scholarly journals listed by ISI, we evaluate the importance of other reference materials to these scholarly journals. For example, we can see which newspapers have the greatest influence on the scholarly literature: New York Times, The Guardian, Wall Street Journal, Washington Post, London Times, Miami Herald, Financial Times, Le Monde, Boston Globe, and Los Angeles Times.
We can view the Eigenfactor score of a journal as a rough estimate of how often a journal will be used by scholars. The Eigenfactor algorithm corresponds to a simple model of research in which readers follow citations as they move from journal to journal. The algorithm effectively calculates the trajectory of a hypothetical “random researcher” who behaves as follows. Our random researcher begins by going to the library and selecting a journal article at random. After reading the article, she selects at random one of the citations from the article. She then proceeds to the cited work and reads a random article there. She selects a new citation from this article, and follows that citation to her next journal volume. The researcher does this ad infinitum. Since we lack the time to carry out this experiment in practice, Eigenfactor uses mathematics to simulate this process.
Because our random researcher moves among journals according the citation network that connects them, the frequency with which she visits each journal gives us a measure of that journal’s importance within network of academic citations. Moreover, if real researchers find a sizable fraction of the articles that they read by following citation chains, the amount of time that our random researcher spends with each journal may give us a reasonable estimate of the amount of time that real researchers spend with each journal.
As librarians work to meet increasing subscription prices with increasingly constrained subscription budgets, powerful measures of journal influence and journal value may usefully supplement expert opinion and other sources of information in making difficult decisions about journal holdings. Our aim with the Eigenfactor project is to provide such a resource to the library community.
Notes
1. P. L. K. Gross and E. M. Gross, “College Libraries and Chemical Education,” Science 66 (1927): 385-389.
2. R. Monastersky, “The number that’s devouring science,” Chronicle of Higher Education 52 (2005) A12.
3. D. J. de Solla Price, “Networks of Scientific Papers,” Science 149 (1965): 510-515
4. Ben Althouse, Martin Rosvall, and Jevin West at the University of Washington, and Ted Bergstrom at the University of California- Santa Barbara.
5. www.eigenfactor.org/methods.pdf; see also previous iterative algorithms proposed by Pinski and Narin 1978;Liebowitz and Palmer 1984 Kalaitzidakis, et al. 2003; Palacios-Huerta 2004; Kodryzcky 2005; Bollen 2006]
References
J. Bollen, M. A. Rodriguez and H. Van de Sompel. 2006. Journal Status. Scientometrics 69: 669-687.
P. Kalaitzidakis and T. Stegnos and T. P. Mamuneas. 2003. Rankings of academic journals and institutions in economics. Journal of the European Economic Association 1: 1346-1366.
Y. K. Kodrzycki and P. D. Yu. 2005. New approaches to ranking economic journals. Federal Reserve Bank of Boston Working Paper: 5-12.
S. J. Liebowitz and J. P. Palmer. 1984. Assessing the relative impacts of economics journals. Journal of Economic Literature 22: 77-88.
I. Palacios-Huerta and O. Volij. 2004. The measurement of intellectual influence. Econometrica 72: 963-977.
G. Pinski and F. Narin. 1976. Citation Influence for Journal Aggregates of Scientific Publications: Theory, with Application to the Literature of Physics. Information Processing and Management 12: 297-312.
Carl Bergstrom is associate professor in the department of biology at the University of Washington, e-mail: cbergst@u.washington.edu
© 2007 Carl Bergstrom