Retrospective analysis of the quality of reports by author-suggested and non-author-suggested reviewers in journals operating on open or single-blind peer review models
Open Access
Abstract
Objectives: To assess whether reports from reviewers recommended by authors show a bias in quality and recommendation for editorial decision, compared with reviewers suggested by other parties, and whether reviewer reports for journals operating on open or single-blind peer review models differ with regard to report quality and reviewer recommendations.Design: Retrospective analysis of the quality of reviewer reports using an established Review Quality Instrument, and analysis of reviewer recommendations and author satisfaction surveys.Setting: BioMed Central biology and medical journals.BMC Infectious DiseasesandBMC Microbiologyare similar in size, rejection rates, impact factors and editorial processes, but the former uses open peer review while the latter uses single-blind peer review. TheJournal of Inflammationhas operated under both peer review models.Sample: Two hundred reviewer reports submitted toBMC Infectious Diseases, 200 reviewer reports submitted toBMC Microbiologyand 400 reviewer reports submitted to theJournal of Inflammation.Results: For each journal, author-suggested reviewers provided reports of comparable quality to non-author-suggested reviewers, but were significantly more likely to recommend acceptance, irrespective of the peer review model (pBMC Infectious Diseases, BMC Microbiologyand theJournal of Inflammation). ForBMC Infectious Diseases, the overall quality of reviewer reports measured by the Review Quality Instrument was 5% higher than forBMC Microbiology(p=0.042). For theJournal of Inflammation, the quality of reports was the same irrespective of the peer review model used.Conclusions: Reviewers suggested by authors provide reports of comparable quality to non-author-suggested reviewers, but are significantly more likely to recommend acceptance. Open peer review reports forBMC Infectious Diseaseswere of higher quality than single-blind reports forBMC Microbiology.There was no difference in quality of peer review in theJournal of Inflammationunder open peer review compared with single blind.Keywords
This publication has 14 references indexed in Scilit:
- Peering into Peer-ReviewThe Journal of Pediatrics, 2011
- Do Author-Suggested Reviewers Rate Submissions More Favorably than Editor-Suggested Reviewers? A Study on Atmospheric Chemistry and PhysicsPLOS ONE, 2010
- A Comparison of Reviewers Selected by Editors and Reviewers Suggested by AuthorsThe Journal of Pediatrics, 2007
- Are reviewers suggested by authors as good as those chosen by editors? Results of a rater-blinded, retrospective studyBMC Medicine, 2006
- Peer review: a flawed process at the heart of science and journalsJournal of the Royal Society of Medicine, 2006
- Differences in Review Quality and Recommendations for Publication Between Peer Reviewers Suggested by Authors or by EditorsJAMA, 2006
- Reviewer selection: author or editor knows best?Thorax, 2005
- A comparison of reports from referees chosen by authors or journal editors in the peer review process.2000
- Open peer review: A randomised controlled trialThe British Journal of Psychiatry, 2000
- Development of the Review Quality Instrument (RQI) for Assessing Peer Reviews of ManuscriptsJournal of Clinical Epidemiology, 1999