Sensitivity of a deep-learning-based breast cancer risk prediction model
Open Access
- 17 April 2025
- journal article
- Published by IOP Publishing in Physics in Medicine & Biology
- Vol. 70 (8) , 085014
Abstract
Objective. When it comes to the implementation of deep-learning based breast cancer risk (BCR) prediction models in clinical settings, it is important to be aware that these models could be sensitive to various factors, especially those arising from the acquisition process. In this work, we investigated how sensitive the state-of-the-art BCR prediction model is to realistic image alterations that can occur as a result of different positioning during the acquisition process. Approach. 5076 mammograms (1269 exams, 650 participants) from the Slovenian and Belgium (University Hospital Leuven) Breast Cancer Screening Programs were collected. The Original MIRAI model was used for 1–5 year BCR estimation. First, BCR was predicted for the original mammograms, which were not changed. Then, a series of different image alteration techniques was performed, such as swapping left and right breasts, removing tissue below the inframammary fold, translations, cropping, rotations, registration and pectoral muscle removal. In addition, a subset of 81 exams, where at least one of the mammograms had to be retaken due to inadequate image quality, served as an approximation of a test–retest experiment. Bland–Altman plots were used to determine prediction bias and 95% limits of agreement (LOA). Additionally, the mean absolute difference in BCR (Mean AD) was calculated. The impact on the overall discrimination performance was evaluated with the AUC. Results. Swapping left and right breasts had no impact on the predicted BCR. The removal of skin tissue below the inframammary fold had minimal impact on the predicted BCR (1–5 year LOA: [−0.02, 0.01]). The model was sensitive to translation, rotation, registration, and cropping, where LOAs of up to ±0.1 were observed. Partial pectoral muscle removal did not have a major impact on predicted BCR, while complete removal of pectoral muscle introduced substantial prediction bias and LOAs (1 year LOA: [−0.07, 0.04], 5 year LOA: [−0.06, 0.03]). The approximation of a real test–retest experiment resulted in LOAs similar to those of simulated image alterations. None of the alterations impacted the overall BCR discrimination performance; the initial 1 year AUC (0.90 [0.88, 0.92]) and 5 year AUC (0.77 [0.75, 0.80]) remained unchanged. Significance. While tested image alterations do not impact overall BCR discrimination performance, substantial changes in predicted 1–5 year BCR can occur on an individual basis.Keywords
Funding Information
- the Slovenian Research and Innovation Agency (P1-0389)
- Research Foundation – Flanders (G0A7121N)
This publication has 58 references indexed in Scilit:
- Generalized Out-of-Distribution Detection: A SurveyInternational Journal of Computer Vision, 2024
- Adversarial attacks in radiology – A systematic reviewEuropean Journal of Radiology, 2023
- Medical image data augmentation: techniques, comparisons and interpretationsArtificial Intelligence Review, 2023
- Pectoralis major muscle sarcoma masquerading breast lesion: A rare case report with review of literatureRadiology Case Reports, 2023
- Test-retest reproducibility of a deep learning–based automatic detection algorithm for the chest radiographEuropean Radiology, 2020
- Grad-CAM: Visual Explanations from Deep Networks via Gradient-Based LocalizationInternational Journal of Computer Vision, 2019
- Visualizing and Understanding Convolutional Networks2014
- Quality assurance and quality control in mammography: a review of available guidance worldwideInsights into Imaging, 2013
- Evaluation of the pectoral muscle in mammography images: The Australian experienceEuropean Journal of Radiography, 2009
- Axillary lymph nodes and breast cancer. A reviewCancer, 1995