The effect of machine learning tools for evidence synthesis on resource use and time-to-completion: protocol for a retrospective pilot study
Open Access
- 17 January 2023
- journal article
- research article
- Published by Springer Nature in Systematic Reviews
- Vol. 12 (1) , 1-8
Abstract
Background: Machine learning (ML) tools exist that can reduce or replace human activities in repetitive or complex tasks. Yet, ML is underutilized within evidence synthesis, despite the steadily growing rate of primary study publication and the need to periodically update reviews to reflect new evidence. Underutilization may be partially explained by a paucity of evidence on how ML tools can reduce resource use and time-to-completion of reviews. Methods: This protocol describes how we will answer two research questions using a retrospective study design: Is there a difference in resources used to produce reviews using recommended ML versus not using ML, and is there a difference in time-to-completion? We will also compare recommended ML use to non-recommended ML use that merely adds ML use to existing procedures. We will retrospectively include all reviews conducted at our institute from 1 August 2020, corresponding to the commission of the first review in our institute that used ML. Conclusion: The results of this study will allow us to quantitatively estimate the effect of ML adoption on resource use and time-to-completion, providing our organization and others with better information to make high-level organizational decisions about ML.Keywords
All Related Versions
Funding Information
- Norwegian Institute of Public Health
This publication has 29 references indexed in Scilit:
- Still moving toward automation of the systematic review process: a summary of discussions at the third meeting of the International Collaboration for Automation of Systematic Reviews (ICASR)Systematic Reviews, 2019
- Software tools for literature screening in systematic reviews in biomedical researchALTEX, 2019
- Living systematic review: 1. Introduction—the why, what, when, and howJournal of Clinical Epidemiology, 2017
- Analysis of the time and workers needed to conduct systematic reviews of medical interventions using data from the PROSPERO registryBMJ Open, 2017
- ROBINS-I: a tool for assessing risk of bias in non-randomised studies of interventionsBMJ, 2016
- Using text mining for study identification in systematic reviews: a systematic review of current approachesSystematic Reviews, 2015
- How Quickly Do Systematic Reviews Go Out of Date? A Survival AnalysisAnnals of Internal Medicine, 2007
- The Science of Reviewing ResearchaAnnals of the New York Academy of Sciences, 1993
- A comparison of results of meta-analyses of randomized control trials and recommendations of clinical experts. Treatments for myocardial infarction.1992
- A Comparison of Results of Meta-analyses of Randomized Control Trials and Recommendations of Clinical ExpertsJAMA, 1992