Abstract
Background The yearly number of publications in many fields, including research on potential health effects of electromagnetic fields (EMF), has steadily risen, making systematic reviews (SRs) increasingly laborious. Automating the title and abstract screening phase using large language models (LLMs) is a promising solution. However, prior studies have underexplored structured LLM workflows, including explicit reasoning outputs, few-shot prompting, and task-decomposition. Methods We systematically devised and evaluated multiple LLM-based workflows using both open-source and proprietary models. The most promising workflows were validated on eight independent SR datasets, focusing on EMF and non-EMF research domains. Workflows were designed to elicit PECOS-wise decisions with explicit reasoning, and record-level verdicts were derived using deterministic decision rules. Results Three workflows, one based on an open-source (llama3.3:70b) and two based on proprietary LLMs, achieved high sensitivity while substantially reducing manual screening effort. The proprietary LLM gpt-4.1-mini embedded in a few-shot workflow with PECOS-wise reasoning, reached a pooled sensitivity of 96.4% (95% CI: 94.8%–97.6%) with a workload reduction of 80.4% (95% CI: 63.7%–90.5%) at an approximate batch API cost of $1.0 per 1,000 records. Overall, small “mini” models performed competitively, demonstrating cost-efficient and resource-friendly applicability. Conclusions Carefully selected LLM workflows implementing few-shot prompting and explicit PECOS-wise reasoning can achieve near-human sensitivity while substantially reducing manual screening effort. These findings provide a transparent and versatile framework for AI-assisted literature screening in systematic reviews, although further optimisation is needed to improve specificity, with executable code for the best workflows available at https://github.com/openbfs/ai-sr-litscreen.

This publication has 41 references indexed in Scilit: