Blinded vs. unblinded peer review of manuscripts submitted to a dermatology journal: a randomized multi-rater study
Open Access
- 28 May 2011
- journal article
- research article
- Published by Oxford University Press (OUP) in British Journal of Dermatology
- Vol. 165 (3) , 563-567
Abstract
Background Submissions to medical and scientific journals are vetted by peer review, but peer review itself has been poorly studied until recently. One concern has been that manuscript reviews in which the reviewer is unblinded (e.g. knows author identity) may be biased, with an increased likelihood that the evaluation will not be strictly on scientific merits. Objectives The purpose of this study was to compare the outcomes of blinded and unblinded reviews of manuscripts submitted to a single dermatology journal via a randomized multi‐rater study. Materials and methods Forty manuscripts submitted to the journal Dermatologic Surgery were assessed by four reviewers, two of whom were randomly selected to be blinded and two unblinded regarding the identities of the manuscripts’ authors. The primary outcome measure was the initial score assigned to each manuscript by each reviewer characterized on an ordinal scale of 1–3, with 1 = accept; 2 = revise (i.e. minor or major revisions) and 3 = reject. Subgroup analysis compared the primary outcome measure across manuscripts from U.S. corresponding authors and foreign corresponding authors. The secondary outcome measure was word count of the narrative portion (i.e. comments to editor and comments to authors) of the reviewer forms. Results There was no significant difference between the scores given to manuscripts by unblinded reviewers and blinded reviewers, both for manuscripts from the U.S. and for foreign submissions. There was also no difference in word count between unblinded and blinded reviews. Conclusions It seems, at least in the case of one dermatology journal, that blinding during peer review does not appear to affect the disposition of the manuscript. To the extent that review word count is a proxy for review quality, there appears to be no quality difference associated with blinding.Keywords
This publication has 9 references indexed in Scilit:
- An Introduction to Categorical Data Analysis2006
- Effect of Blinded Peer Review on Abstract AcceptanceJAMA, 2006
- The Fallacy of Double-Blinded Peer ReviewAmerican Journal of Roentgenology, 2003
- Incidence and Nature of Unblinding by Authors: Our Experience at Two Radiology Journals with Double-Blinded Peer Review PoliciesAmerican Journal of Roentgenology, 2002
- Blinded ReviewsChest, 1998
- Effect on the Quality of Peer Review of Blinding Reviewers and Asking Them to Sign Their ReportsJAMA, 1998
- Effect of Blinding and Unmasking on the Quality of Peer ReviewJAMA, 1998
- Masking, Blinding, and Peer Review: The Blind Leading the BlindedAnnals of Internal Medicine, 1998
- How blind is blind review?American Journal of Public Health, 1991