Attention-Based Deep Learning System for Classification of Breast Lesions—Multimodal, Weakly Supervised Approach
Open Access
Abstract
Breast cancer is the most frequent female cancer, with a considerable disease burden and high mortality. Early diagnosis with screening mammography might be facilitated by automated systems supported by deep learning artificial intelligence. We propose a model based on a weakly supervised Clustering-constrained Attention Multiple Instance Learning (CLAM) classifier able to train under data scarcity effectively. We used a private dataset with 1174 non-cancer and 794 cancer images labelled at the image level with pathological ground truth confirmation. We used feature extractors (ResNet-18, ResNet-34, ResNet-50 and EfficientNet-B0) pre-trained on ImageNet. The best results were achieved with multimodal-view classification using both CC and MLO images simultaneously, resized by half, with a patch size of 224 px and an overlap of 0.25. It resulted in AUC-ROC = 0.896 ± 0.017, F1-score 81.8 ± 3.2, accuracy 81.6 ± 3.2, precision 82.4 ± 3.3, and recall 81.6 ± 3.2. Evaluation with the Chinese Mammography Database, with 5-fold cross-validation, patient-wise breakdowns, and transfer learning, resulted in AUC-ROC 0.848 ± 0.015, F1-score 78.6 ± 2.0, accuracy 78.4 ± 1.9, precision 78.8 ± 2.0, and recall 78.4 ± 1.9. The CLAM algorithm’s attentional maps indicate the features most relevant to the algorithm in the images. Our approach was more effective than in many other studies, allowing for some explainability and identifying erroneous predictions based on the wrong premises.Keywords
Funding Information
- European Commission (952103, 2021/05/X/NZ5/01776, 23/2022/IDUB/III.1/Ra, 24/2022/IDUB/III.1/Ra, 26/1/2022/IDUB/III.4.1/Tc)
- National Science Center (NCN), Poland (952103, 2021/05/X/NZ5/01776, 23/2022/IDUB/III.1/Ra, 24/2022/IDUB/III.1/Ra, 26/1/2022/IDUB/III.4.1/Tc)
- GUT IDUB RADIUM (952103, 2021/05/X/NZ5/01776, 23/2022/IDUB/III.1/Ra, 24/2022/IDUB/III.1/Ra, 26/1/2022/IDUB/III.4.1/Tc)
- GUT IDUB TECHNETIUM (952103, 2021/05/X/NZ5/01776, 23/2022/IDUB/III.1/Ra, 24/2022/IDUB/III.1/Ra, 26/1/2022/IDUB/III.4.1/Tc)
This publication has 68 references indexed in Scilit:
- Explaining nonlinear classification decisions with deep Taylor decompositionPattern Recognition, 2017
- Evaluating the Visualization of What a Deep Neural Network Has LearnedIEEE Transactions on Neural Networks and Learning Systems, 2016
- Clinical Diagnosis and Management of Breast CancerJournal of Nuclear Medicine, 2016
- Multiple-Instance Learning for Anomaly Detection in Digital MammographyIEEE Transactions on Medical Imaging, 2016
- Multiple Instance Learning2016
- On Pixel-Wise Explanations for Non-Linear Classifier Decisions by Layer-Wise Relevance PropagationPLOS ONE, 2015
- ImageNet Large Scale Visual Recognition ChallengeInternational Journal of Computer Vision, 2015
- Comparison between different imaging techniques in the evaluation of malignant breast lesions: can 3D ultrasound be useful?La radiologia medica, 2013
- INbreastAcademic Radiology, 2012
- Digital Mammographic Artifacts on Full-Field Systems: What Are They and How Do I Fix Them?RadioGraphics, 2008