The effect of machine learning tools for evidence synthesis on resource use and time-to-completion: protocol for a retrospective pilot study
Authors:
- Ashley Elizabeth MullerAMAshley Elizabeth Muller
- Rigor C BergRBRigor C Berg
- Jose Francisco Meneses-EschavezJMJose Francisco Meneses-Eschavez
- Patricia Sofia Jacobsen JardimPJPatricia Sofia Jacobsen Jardim
- Chris CooperCCChris Cooper
- Christopher James RoseCRChristopher James Rose
Preprint
- 23 February 2023
- preprint
- Published by Springer Nature in Research Square
Abstract
Background Machine learning (ML) tools exist that can reduce or replace human activities in repetitive or complex tasks. Yet ML is underutilized within evidence synthesis, despite the steadily growing rate of primary study publication and need to periodically update reviews to reflect new evidence. Underutilization may be partially explained by a paucity of evidence on how ML tools can reduce resource use and time-to-completion of reviews. Methods This protocol describes how we will answer two research questions using a retrospective study design: Is there a difference in resources used to produce reviews using recommended ML versus not using ML, and is there a difference in time-to-completion? We will also compare recommended ML use to non-recommended ML use. We will retrospectively include all reviews conducted at our institute from 1 August 2020, corresponding to the commission of the first review in our institute that used ML. We will use the results from this study to design a rigorous, multi-institutional, prospective study that will additionally explore review quality. Conclusion We invite other evidence synthesis groups to adopt and adapt this protocol and to collaborate with us.Keywords
All Related Versions
This publication has 27 references indexed in Scilit:
- Still moving toward automation of the systematic review process: a summary of discussions at the third meeting of the International Collaboration for Automation of Systematic Reviews (ICASR)Systematic Reviews, 2019
- Living systematic review: 1. Introduction—the why, what, when, and howJournal of Clinical Epidemiology, 2017
- AMSTAR 2: a critical appraisal tool for systematic reviews that include randomised or non-randomised studies of healthcare interventions, or bothBMJ, 2017
- Analysis of the time and workers needed to conduct systematic reviews of medical interventions using data from the PROSPERO registryBMJ Open, 2017
- ROBINS-I: a tool for assessing risk of bias in non-randomised studies of interventionsBMJ, 2016
- Using text mining for study identification in systematic reviews: a systematic review of current approachesSystematic Reviews, 2015
- How Quickly Do Systematic Reviews Go Out of Date? A Survival AnalysisAnnals of Internal Medicine, 2007
- The Science of Reviewing ResearchaAnnals of the New York Academy of Sciences, 1993
- Cryptographic limitations on learning Boolean formulae and finite automata1993
- A Comparison of Results of Meta-analyses of Randomized Control Trials and Recommendations of Clinical ExpertsJAMA, 1992