A Community's Perspective on the Status and Future of Peer Review in Software Engineering
Preprint
Abstract
Context: Pre-publication peer review of scientific articles is considered a key element of the research process in software engineering, yet it is often perceived as not to work fully well. Objective: We aim at understanding the perceptions of and attitudes towards peer review of authors and reviewers at one of software engineering's most prestigious venues, the International Conference on Software Engineering (ICSE). Method: We invited 932 ICSE 2014/15/16 authors and reviewers to participate in a survey with 10 closed and 9 open questions. Results: We present a multitude of results, such as: Respondents perceive only one third of all reviews to be good, yet one third as useless or misleading; they propose double-blind or zero-blind reviewing regimes for improvement; they would like to see showable proofs of (good) reviewing work be introduced; attitude change trends are weak. Conclusion: The perception of the current state of software engineering peer review is fairly negative. Also, we found hardly any trend that suggests reviewing will improve by itself over time; the community will have to make explicit efforts. Fortunately, our (mostly senior) respondents appear more open for trying different peer reviewing regimes than we had expected.Keywords
All Related Versions
This publication has 18 references indexed in Scilit:
- Peer review motivation frames: A qualitative approachEuropean Management Journal, 2016
- Rewarding reviewers – sense or sensibility? A Wiley study explainedLearned Publishing, 2016
- Peer review: The current landscape and future trendsLearned Publishing, 2016
- Growth rates of modern science: A bibliometric analysis based on the number of publications and cited referencesJournal of the Association for Information Science and Technology, 2015
- Peer review in a changing world: An international study measuring the attitudes of researchersJournal of the American Society for Information Science and Technology, 2012
- Peerage of Science: will it work?Trends in Ecology & Evolution, 2012
- Double-blind review favours increased representation of female authorsTrends in Ecology & Evolution, 2008
- The Construction of Attitudes2001
- Peer review for journals: Evidence on quality control, fairness, and innovationScience and Engineering Ethics, 1997
- A Citation Analysis of the Impact of Blinded Peer ReviewJAMA, 1994