Cost-Efficient Large Language Model-Assisted Title and Abstract Screening for Systematic Reviews: Method Evaluation and Validation
Preprint
- 4 September 2026
- preprint
- Published by Springer Nature in Research Square
Abstract
Background The yearly number of publications in many fields, including research on potential health effects of electromagnetic fields (EMF), has steadily risen, making systematic reviews (SRs) increasingly laborious. Automating the title and abstract screening phase using large language models (LLMs) is a promising solution. However, prior studies have underexplored structured LLM workflows, including explicit reasoning outputs, few-shot prompting, and task-decomposition. Methods We systematically devised and evaluated multiple LLM-based workflows using both open-source and proprietary models. The most promising workflows were validated on eight independent SR datasets, focusing on EMF and non-EMF research domains. Workflows were designed to elicit PECOS-wise decisions with explicit reasoning, and record-level verdicts were derived using deterministic decision rules. Results Three workflows, one based on an open-source (llama3.3:70b) and two based on proprietary LLMs, achieved high sensitivity while substantially reducing manual screening effort. The proprietary LLM gpt-4.1-mini embedded in a few-shot workflow with PECOS-wise reasoning, reached a pooled sensitivity of 96.4% (95% CI: 94.8%–97.6%) with a workload reduction of 80.4% (95% CI: 63.7%–90.5%) at an approximate batch API cost of $1.0 per 1,000 records. Overall, small “mini” models performed competitively, demonstrating cost-efficient and resource-friendly applicability. Conclusions Carefully selected LLM workflows implementing few-shot prompting and explicit PECOS-wise reasoning can achieve near-human sensitivity while substantially reducing manual screening effort. These findings provide a transparent and versatile framework for AI-assisted literature screening in systematic reviews, although further optimisation is needed to improve specificity, with executable code for the best workflows available at https://github.com/openbfs/ai-sr-litscreen.Keywords
All Related Versions
This publication has 41 references indexed in Scilit:
- Position Statement on Artificial Intelligence (AI) Use in Evidence Synthesis Across Cochrane, the Campbell Collaboration, JBI, and the Collaboration for Environmental Evidence 2025Campbell Systematic Reviews, 2025
- The effects of radiofrequency electromagnetic field exposure on biomarkers of oxidative stress in vivo and in vitro: A systematic review of experimental studiesEnvironment International, 2024
- The effect of exposure to radiofrequency electromagnetic fields on cognitive performance in human experimental studies: Systematic review and meta-analysesEnvironment International, 2024
- Performance of a Large Language Model in Screening CitationsJAMA Network Open, 2024
- Effects of radiofrequency electromagnetic field (RF-EMF) exposure on male fertility: A systematic review of experimental studies on non-human mammals and human sperm in vitroEnvironment International, 2024
- The effects of radiofrequency electromagnetic fields exposure on tinnitus, migraine and non-specific symptoms in the general and working population: A systematic review and meta-analysis on human observational studiesEnvironment International, 2023
- Efficacy and safety of selective serotonin reuptake inhibitors in COVID-19 management: a systematic review and meta-analysisClinical Microbiology and Infection, 2023
- The effect of exposure to radiofrequency fields on cancer risk in the general and working population: A protocol for a systematic review of human observational studiesEnvironment International, 2021
- Long-term Outcomes of Cognitive Behavioral Therapy for Anxiety-Related DisordersJAMA Psychiatry, 2020
- Identifying the PECO: A framework for formulating good questions to explore the association of environmental and other exposures with health outcomesEnvironment International, 2018