Improving Robustness and Reliability in Medical Image Classification With Latent-Guided Diffusion and Nested-Ensembles
- 30 June 2025
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Medical Imaging
- Vol. 44 (12) , 4890-4902
Abstract
Once deployed, medical image analysis methods are often faced with unexpected image corruptions and noise perturbations. These unknown covariate shifts present significant challenges to deep learning based methods trained on “clean” images. This often results in unreliable predictions and poorly calibrated confidence, hence hindering clinical applicability. While recent methods have been developed to address specific issues such as confidence calibration or adversarial robustness, no single framework effectively tackles all these challenges simultaneously. To bridge this gap, we propose LaDiNE, a novel ensemble learning method combining the robustness of Vision Transformers with diffusion-based generative models for improved reliability in medical image classification. Specifically, transformer encoder blocks are used as hierarchical feature extractors that learn invariant features from images for each ensemble member, resulting in features that are robust to input perturbations. In addition, diffusion models are used as flexible density estimators to estimate member densities conditioned on the invariant features, leading to improved modeling of complex data distributions while retaining properly calibrated confidence. Extensive experiments on tuberculosis chest X-rays and melanoma skin cancer datasets demonstrate that LaDiNE achieves superior performance compared to a wide range of state-of-the-art methods by simultaneously improving prediction accuracy and confidence calibration under unseen noise, adversarial perturbations, and resolution degradation.Keywords
All Related Versions
Funding Information
- Natural Sciences and Engineering Research Council of Canada
- Canadian Institute for Advanced Research (CIFAR) Artificial Intelligence Chairs Program
- Mila—Quebec Artificial Intelligence Institute Technology Transfer Program
- Mila—Google Research Grant
- Calcul Québec
- Alliance de recherche numérique du Canada
- Canada First Research Excellence Fund, awarded to the Healthy Brains, Healthy Lives initiative at McGill University
This publication has 46 references indexed in Scilit:
- Mitigating Calibration Bias Without Fixed Attribute Grouping for Improved Fairness in Medical Imaging Analysis2023
- SADM: Sequence-Aware Diffusion Model for Longitudinal Medical Image Generation2023
- Brain Imaging Generation with Latent Diffusion Models2022
- Measurement-Conditioned Denoising Diffusion Probabilistic Model for Under-Sampled Medical Image Reconstruction2022
- Self-Ensembling Vision Transformer (SEViT) for Robust Medical Image Classification2022
- A Robust Volumetric Transformer for Accurate 3D Tumor Segmentation2022
- Building robust pathology image analyses with uncertainty quantificationComputer Methods and Programs in Biomedicine, 2021
- Diagnostic Assessment of Deep Learning Algorithms for Detection of Lymph Node Metastases in Women With Breast CancerJAMA, 2017
- Development and Validation of a Deep Learning Algorithm for Detection of Diabetic Retinopathy in Retinal Fundus PhotographsJAMA, 2016
- Bagging predictorsMachine Learning, 1996