A deep learning system for non-invasive breast cancer diagnosis with multimodal data
- 19 May 2026
- journal article
- Published by Springer Nature in Nature Biomedical Engineering
Abstract
Early and accurate diagnosis of breast cancer is critical for minimizing needle biopsies and enhancing patient outcomes and requires effective integration of multimodal information. In this article, we introduce a breast cancer intelligent non-invasive diagnosis system (BINDS) to integrate multimodal medical imaging data for breast cancer risk assessment and subtype classification. BINDS uses a two-stage diagnostic approach to match the clinical workflow, where an initial assessment with ultrasound and/or mammography is performed, followed by a more comprehensive multimodal diagnosis incorporating magnetic resonance imaging. In addition, a new radiology-pathology alignment mechanism is proposed to facilitate extraction of pathology-relevant features from radiological images. BINDS is developed and validated with a diverse dataset of 27,048 participants from 8 centres and 7 public datasets. Importantly, BINDS supports flexible combinations of input modalities during training and validation. Notably, BINDS attains an area under the receiver operating characteristic curve of 0.973, and can assist radiologists in reducing biopsies of benign lesions by up to 32.4%. These findings highlight the potential of BINDS to advance breast cancer diagnosis by enabling precise and adaptable decision-making across diverse clinical scenarios and resource settings.Keywords
This publication has 73 references indexed in Scilit:
- IRUM: An Image Representation and Unified Learning Method for Breast Cancer Diagnosis from Multi-View Ultrasound Images2024
- BUS‐BRA: A breast ultrasound dataset for assessing computer‐aided diagnosis systemsMedical Physics, 2023
- Deep learning-based system for automatic prediction of triple-negative breast cancer from ultrasound imagesMedical & Biological Engineering & Computing, 2022
- Domain Generalization for Mammography Detection via Multi-style and Multi-view Contrastive Learning2021
- A deep learning model integrating mammography and clinical factors facilitates the malignancy prediction of BI-RADS 4 microcalcifications in breast cancer screeningEuropean Radiology, 2021
- Prediction of breast cancer molecular subtypes on DCE-MRI using convolutional neural network with transfer learning between two centersEuropean Radiology, 2020
- Diagnosis of Benign and Malignant Breast Lesions on DCE‐MRI by Using Radiomics and Deep Learning With Consideration of Peritumor TissueJournal of Magnetic Resonance Imaging, 2019
- An overview of mammographic density and its association with breast cancerBreast Cancer, 2018
- Breast cancer diagnosis: Imaging techniques and biochemical markersJournal of Cellular Physiology, 2018
- Combined Screening With Ultrasound and Mammography vs Mammography Alone in Women at Elevated Risk of Breast CancerJAMA, 2008