Impact factor volatility due to a single paper: A comprehensive analysis
Open Access
- 1 June 2020
- journal article
- research article
- Published by MIT Press in Quantitative Science Studies
- Vol. 1 (2) , 639-663
Abstract
We study how a single paper affects the impact factor (IF) of a journal by analyzing data from 3,088,511 papers published in 11639 journals in the 2017 Journal Citation Reports of Clarivate Analytics. We find that IFs are highly volatile. For example, the top-cited paper of 381 journals caused their IF to increase by more than 0.5 points, while for 818 journals the relative increase exceeded 25%. One in 10 journals had their IF boosted by more than 50% by their top three cited papers. Because the single-paper effect on the IF is inversely proportional to journal size, small journals are rewarded much more strongly than large journals for a highly cited paper, while they are penalized more for a low-cited paper, especially if their IF is high. This skewed reward mechanism incentivizes high-IF journals to stay small to remain competitive in rankings. We discuss the implications for breakthrough papers appearing in prestigious journals. We question the reliability of IF rankings given the high IF sensitivity to a few papers that affects thousands of journals.Keywords
All Related Versions
This publication has 32 references indexed in Scilit:
- Implications of a Single Highly Cited Article on a Journal and Its Citation Indexes: A Tale of Two JournalsAccountability in Research, 2013
- The use of percentiles and percentile rank classes in the analysis of bibliometric data: Opportunities and limitsJournal of Informetrics, 2013
- Towards a new crown indicator: an empirical analysisScientometrics, 2011
- Further steps towards an ideal method of measuring citation performance: The avoidance of citation (ratio) averages in field-normalizationJournal of Informetrics, 2011
- Metrics: journal's impact factor skewed by a single paperNature, 2010
- The European Research Council Takes FlightCell, 2009
- Editorial: Is PRL Too Large to Have an “Impact”?Physical Review Letters, 2009
- Citation StatisticsStatistical Science, 2009
- Escape from the impact factorEthics in Science and Environmental Politics, 2008
- Show me the dataJournal of Cell Biology, 2007