Systematic reviewers' perspectives on replication of systematic reviews: A survey
Authors:
Open Access
- 10 April 2023
- journal article
- review article
- Published by Wiley in Cochrane Evidence Synthesis and Methods
- Vol. 1 (2)
Abstract
Background: Replication is essential to the scientific method. It is unclear what systematic reviewers think about the replication of systematic reviews (SRs). Therefore, we aimed to explore systematic reviewers' perspectives on (a) the definition and importance of SR replication; (b) incentives and barriers to conducting SR replication; and (c) a checklist to guide when to replicate an SR.Methods: We searched PubMed for SRs published from January to April 2021, from which we randomly allocated 50% to this survey and 50% to another survey on data sharing in SRs. We sent an electronic survey to authors of these SRs (n = 4669) using Qualtrics. Quantitative responses were summarized using frequency analysis. Free‐text answers were coded using an inductive approach.Results: The response rate was 9% (n = 409). Most participants considered “replication of SRs” as redoing an SR (68%) or reanalyzing originally collected data (61%), using the same or similar methods. Participants also considered updating an SR, either one's own (42%) or others (43%), equivalent to replication. Most participants agreed that replication of SRs is important (89%). Although 54% of participants reported having conducted a replication of a SR, only 22% have published a replication within 5 years. Those who published a replication (n = 89) often found their replication supported (47%) or expanded the generalizability of the original review (51%). The most common perceived barriers to replicating SRs were difficulty publishing (75%), less prestige (65%), fewer citations (56%), and less impact on career advancement (55%) compared to conducting an original SR. A checklist to assess the need for replication was deemed useful (79%) and easy to apply in practice (69%) by participants.Conclusion: Reviewers have various perceptions of what constitutes a replication of SRs. Reviewers see replication as important and valuable but perceive several barriers to conducting replications. Institutional support should be better communicated to reviewers to address these perceptions.Keywords
Funding Information
- Australian Research Council (DE200101618)
This publication has 44 references indexed in Scilit:
- The Policy Role of Systematic Reviews: Past, Present and FutureSpringer Science Reviews, 2014
- Facts Are More Important Than NoveltyEducational Researcher, 2014
- Understanding replication of experiments in software engineering: A classificationInformation and Software Technology, 2014
- Accumulating Research: A Systematic Account of How Cumulative Meta-Analyses Would Have Provided Knowledge, Improved Health, Reduced Harm and Saved ResourcesPLOS ONE, 2014
- Replications in Psychology ResearchPerspectives on Psychological Science, 2012
- Editorial: A Call for Replications of Research in Pediatric Psychology and Guidance for AuthorsJournal of Pediatric Psychology, 2010
- A General Inductive Approach for Analyzing Qualitative Evaluation DataAmerican Journal of Evaluation, 2006
- Reflections on replication in OM research and this special issueJournal of Operations Management, 2005
- How Journal Editors View Replication ResearchJournal of Advertising, 1995
- Using Research in PracticeWestern Journal of Nursing Research, 1986