AI-integrated Screening to Replace Double Reading of Mammograms: A Population-wide Accuracy and Feasibility Study
Open Access
- 1 November 2024
- journal article
- research article
- Published by Radiological Society of North America (RSNA) in Radiology: Artificial Intelligence
- Vol. 6 (6) , e230529
Abstract
"Just Accepted" papers have undergone full peer review and have been accepted for publication in Radiology: Artificial Intelligence. This article will undergo copyediting, layout, and proof review before it is published in its final version. Please note that during production of the final copyedited article, errors may be discovered which could affect the content. Mammography screening supported by deep learning-based artificial intelligence (AI) solutions can potentially reduce workload without compromising breast cancer detection accuracy, but the site of deployment in the workflow might be crucial. This retrospective study compared three simulated AI-integrated screening scenarios with standard double reading with arbitration in a sample of 249,402 mammograms from a representative screening population. A commercial AI system replaced the first reader (Scenario 1: Integrated AIfirst), the second reader (Scenario 2: Integrated AIsecond), or both readers for triaging of low- and high-risk cases (Integrated AItriage). AI threshold values were partly chosen based on previous validation and fixing screen-read volume reduction at approximately 50% across scenarios. Detection accuracy measures were calculated. Compared with standard double reading, Integrated AIfirst showed no evidence of a difference in accuracy metrics except for a higher arbitration rate (+0.99%; P < .001). Integrated AIsecond had lower sensitivity (-1.58%; P < 0.001), negative predictive value (NPV) (- 0.01%; P < .001) and recall rate (< 0.06%; P = 0.04), but a higher positive predictive value (PPV) (+0.03%; P < .001) and arbitration rate (+1.22%; P < .001). Integrated AItriage achieved higher sensitivity (+1.33%; P < .001), PPV (+0.36%; P = .03), and NPV (+0.01%; P < .001) but lower arbitration rate (-0.88%; P < .001). Replacing one or both readers with AI seems feasible, however, the site of application in the workflow can have clinically relevant effects on accuracy and workload. ©RSNA, 2024.Keywords
Funding Information
- Innovation Fund (10240300)
This publication has 18 references indexed in Scilit:
- Population-wide evaluation of artificial intelligence and radiologist assessment of screening mammogramsEuropean Radiology, 2023
- Artificial intelligence for breast cancer detection in screening mammography in Sweden: a prospective, population-based, paired-reader, non-inferiority studyThe Lancet Digital Health, 2023
- Artificial intelligence-supported screen reading versus standard double reading in the Mammography Screening with Artificial Intelligence trial (MASAI): a clinical safety analysis of a randomised, controlled, non-inferiority, single-blinded, screening accuracy studyThe Lancet Oncology, 2023
- Artificial Intelligence as Supporting Reader in Breast Screening: A Novel Workflow to Preserve Quality and Reduce WorkloadJournal of Breast Imaging, 2023
- Artificial intelligence (AI) for breast cancer screening: BreastScreen population-based cohort study of cancer detectionEBioMedicine, 2023
- Possible strategies for use of artificial intelligence in screen-reading of mammograms, based on retrospective data from 122,969 screening examinationsEuropean Radiology, 2022
- Independent External Validation of Artificial Intelligence Algorithms for Automated Interpretation of Screening Mammography: A Systematic ReviewJournal of the American College of Radiology, 2022
- Machine Learning for Workflow Applications in Screening Mammography: Systematic Review and Meta-AnalysisRadiology, 2021
- Use of artificial intelligence for image analysis in breast cancer screening programmes: systematic review of test accuracyBMJ, 2021
- Comparisons of Predictive Values of Binary Medical Diagnostic Tests for Paired DesignsBiometrics, 2000