Crowdsourcing in peer review: estimating the effects of crowd peer review on academic impact and social media impact
- 20 February 2026
- journal article
- review article
- Published by Emerald Publishing in Online Information Review
- Vol. 50 (3) , 524-540
Abstract
Purpose: Traditional peer review systems face growing criticism for their inefficiency and bias, prompting the exploration of innovative alternatives such as online crowdsourcing-based peer review models. This study aimed to investigate the effects of crowd peer review—specifically, open and closed variants—on both academic impact and social media impact, addressing the need for empirical evidence on these emerging digital practices in scholarly communication. Design/methodology/approach: Articles published by Synlett that implemented closed crowd peer review (n = 917) and articles published by Atmospheric Chemistry and Physics (ACP) that implemented open crowd peer review (n = 12,827) were selected as samples. Causal inference was used to examine the data. Alternative outcome variables and different datasets were used to check the robustness of the results. Findings: The results revealed that open crowd peer review significantly enhanced both academic impact and social media impact, whereas closed crowd peer review had no significant effect on either metric. These findings suggest that openness and transparency mechanisms are necessary rather than optional features of an effective crowdsourcing-based peer review system. Originality/value: This study provides novel empirical evidence on the quantifiable benefits of open crowd peer review, advancing the understanding of digital crowdsourcing mechanisms in academic communication. By highlighting the differential impacts of open and closed models, the results offer actionable insights for publishers and platforms seeking to innovate peer review processes in alignment with digital transformation and online community engagement. Peer review: The peer review history for this article is available at: https://publons.com/publon/10.1108/OIR-06-2025-0426.Keywords
This publication has 53 references indexed in Scilit:
- Knowledge integration and diffusion structures of interdisciplinary research: A large‐scale analysis based on propensity score matchingJournal of the Association for Information Science and Technology, 2025
- Impact of author characteristics on outcomes of single- versus double-blind peer review: a systematic review of comparative studies in scientific abstracts and publicationsScientometrics, 2024
- Do open science badges work? Estimating the effects of open science badges on an article’s social media attention and research impactsScientometrics, 2023
- Altmetric scores: An early indicator of research impactJournal of the Association for Information Science and Technology, 2018
- Schneller und weniger aufwendig begutachtenNachrichten aus der Chemie, 2017
- The ecstasy and the agony of the altmetric scoreScientometrics, 2016
- Peer review in a changing world: An international study measuring the attitudes of researchersJournal of the American Society for Information Science and Technology, 2012
- In public peer review of submitted manuscripts, how do reviewer comments differ from comments written by interested members of the scientific community? A content analysis of comments written for Atmospheric Chemistry and PhysicsScientometrics, 2012
- Closed versus open reviewing of journal manuscripts: how far do comments differ in language use?Scientometrics, 2011
- Is interactive open access publishing able to identify high-impact submissions? A study on the predictive validity of Atmospheric Chemistry and Physics by using percentile rank classesJournal of the American Society for Information Science and Technology, 2010