External Evaluation of a Mammography-based Deep Learning Model for Predicting Breast Cancer in an Ethnically Diverse Population
Authors:
- Olasubomi J. OmoleyeOOOlasubomi J. Omoleye
- Anna E. WoodardAWAnna E. Woodard
- Frederick M. HowardFHFrederick M. Howard
- Fangyuan ZhaoFZFangyuan Zhao
- Toshio F. YoshimatsuTYToshio F. Yoshimatsu
- Yonglan ZhengYZYonglan Zheng
- Alexander T. PearsonAPAlexander T. Pearson
- Maksim LeventalMLMaksim Levental
- Benjamin S. AribisalaBABenjamin S. Aribisala
- Kirti KulkarniKKKirti Kulkarni
- Gregory S. KarczmarGKGregory S. Karczmar
- Olufunmilayo I. OlopadeOOOlufunmilayo I. Olopade
- 1 November 2023
- journal article
- research article
- Published by Radiological Society of North America (RSNA) in Radiology: Artificial Intelligence
- Vol. 5 (6) , e220299
Abstract
To externally evaluate a mammography-based deep learning (DL) model (Mirai) in a high-risk racially diverse population and compare its performance with other mammographic measures. A total of 6435 screening mammograms in 2096 female patients (median age, 56.4 years ± 11.2 [SD]) enrolled in a hospital-based case-control study from 2006 to 2020 were retrospectively evaluated. Pathologically confirmed breast cancer was the primary outcome. Mirai scores were the primary predictors. Breast density and Breast Imaging Reporting and Data System (BI-RADS) assessment categories were comparative predictors. Performance was evaluated using area under the receiver operating characteristic curve (AUC) and concordance index analyses. Mirai achieved 1- and 5-year AUCs of 0.71 (95% CI: 0.68, 0.74) and 0.65 (95% CI: 0.64, 0.67), respectively. One-year AUCs for nondense versus dense breasts were 0.72 versus 0.58 (P = .10). There was no evidence of a difference in near-term discrimination performance between BI-RADS and Mirai (1-year AUC, 0.73 vs 0.68; P = .34). For longer-term prediction (2–5 years), Mirai outperformed BI-RADS assessment (5-year AUC, 0.63 vs 0.54; P < .001). Using only images of the unaffected breast reduced the discriminatory performance of the DL model (P < .001 at all time points), suggesting that its predictions are likely dependent on the detection of ipsilateral premalignant patterns. A mammography DL model showed good performance in a high-risk external dataset enriched for African American patients, benign breast disease, and BRCA mutation carriers, and study findings suggest that the model performance is likely driven by the detection of precancerous changes. Keywords: Breast, Cancer, Computer Applications, Convolutional Neural Network, Deep Learning Algorithms, Informatics, Epidemiology, Machine Learning, Mammography, Oncology, Radiomics Supplemental material is available for this article. © RSNA, 2023 See also commentary by Kontos and Kalpathy-Cramer in this issue.Keywords
Funding Information
- Susan G. Komen for the Cure (SAC210203, TREND21675016)
- National Institutes of Health/National Cancer Institute (P20CA233307)
- Breast Cancer Research Foundation (BCRF-21-071)
This publication has 33 references indexed in Scilit:
- Breast Cancer Screening Using Tomosynthesis and Digital Mammography in Dense and Nondense BreastsJAMA, 2016
- Breast cancer detection using automated whole breast ultrasound and mammography in radiographically dense breastsEuropean Radiology, 2009
- Field effect in cancer-an update.2009
- Mammographic density, breast cancer risk and risk predictionBreast Cancer Research, 2007
- Cancer Yield of Mammography, MR, and US in High-Risk Women: Prospective Multi-Institution Breast Cancer Screening StudyRadiology, 2007
- Mammographic Density and the Risk and Detection of Breast CancerNew England Journal of Medicine, 2007
- Mapping Geographic Zones of Cancer Risk with Epigenetic Biomarkers in Normal Breast TissueClinical Cancer Research, 2006
- Breast Density and Parenchymal Patterns as Markers of Breast Cancer Risk: A Meta-analysisCancer Epidemiology, Biomarkers & Prevention, 2006
- Evaluation of Abnormal Mammography Results and Palpable Breast AbnormalitiesAnnals of Internal Medicine, 2003
- Comparing the Areas under Two or More Correlated Receiver Operating Characteristic Curves: A Nonparametric ApproachBiometrics, 1988