Transformer-based Deep Learning Models with Shape Guidance for Predicting Breast Cancer in Mammography Images
Open Access
- 19 December 2025
- journal article
- Published by Springer Nature in Journal of Imaging Informatics in Medicine
- Vol. 39 (5) , 4038-4051
Abstract
Recent breast cancer research has investigated shape-based attention guidance in Vision Transformer (ViT) models, focusing on anatomical structures and the heterogeneity surrounding tumors. However, few studies have clarified the optimal transformer encoder layer stage for applying attention guidance. Our study aimed to evaluate the effectiveness of shape-guidance strategies by varying the combinations of encoder layers that guide attention to breast structures and by comparing the proposed models with conventional models. For the shape-guidance strategy, we applied breast masks to the attention mechanism to emphasize spatial dependencies and enhance the learning of positional relationships within breast anatomy. We then compared the representative models—Masked Transformer models that demonstrated the best performance across layer combinations—with the conventional ResNet50, ViT, and SwinT V2. In our study, a total of 2,436 publicly available mammography images from the Chinese Mammography Database via The Cancer Imaging Archive were analyzed. Three-fold cross-validation was employed, with a patient-wise split of 70% for training and 30% for validation. Model performance on differentiating breast cancer from non-cancer images was assessed by the area under the receiver-operating characteristic curve (AUROC). The results showed that applying masks at the Shallow and Deep stages gave the highest AUROC for Masked ViT. The Masked ViT achieved an AUROC of 0.885 [95% confidence interval: 0.849–0.918], a sensitivity of 0.876, and a specificity of 0.802, outperforming all other conventional models. These results indicate that incorporating mask guidance into particular Transformer encoders promotes representation learning, highlighting their potential as decision-support tools in breast cancer diagnosis.Keywords
Funding Information
- Advanced Graduate Program for Future Medicine and Health Care
- The Japan Society for the Promotion of Science (JSPS) Grants-in-Aid for Scientific Research (25KJ0573)
- The Yamada Mitsuru Scholarship Foundation
- The Morinokuni Scholarship Foundation
This publication has 43 references indexed in Scilit:
- Breast cancer classification based on breast tissue structures using the Jigsaw puzzle task in self-supervised learningRadiological Physics and Technology, 2025
- Deep learning model for predicting the presence of stromal invasion of breast cancer on digital breast tomosynthesisRadiological Physics and Technology, 2023
- Robust and data-efficient generalization of self-supervised machine learning for diagnostic imagingNature Biomedical Engineering, 2023
- Transformers in medical imaging: A surveyMedical Image Analysis, 2023
- Transformer Based Multi-view Network for Mammographic Image Classification2022
- Intra-class Contrastive Learning Improves Computer Aided Diagnosis of Breast Cancer in Mammography2022
- Convolutional neural networks for breast cancer detection in mammography: A surveyComputers in Biology and Medicine, 2021
- International evaluation of an AI system for breast cancer screeningNature, 2020
- Benign and malignant classification of mammogram images based on deep learningBiomedical Signal Processing and Control, 2019
- Large scale deep learning for computer aided detection of mammographic lesionsMedical Image Analysis, 2017