Concept-Based Lesion Aware Transformer for Interpretable Retinal Disease Diagnosis
- 16 July 2024
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Medical Imaging
- Vol. 44 (1) , 57-68
Abstract
Existing deep learning methods have achieved remarkable results in diagnosing retinal diseases, showcasing the potential of advanced AI in ophthalmology. However, the black-box nature of these methods obscures the decision-making process, compromising their trustworthiness and acceptability. Inspired by the concept-based approaches and recognizing the intrinsic correlation between retinal lesions and diseases, we regard retinal lesions as concepts and propose an inherently interpretable framework designed to enhance both the performance and explainability of diagnostic models. Leveraging the transformer architecture, known for its proficiency in capturing long-range dependencies, our model can effectively identify lesion features. By integrating with image-level annotations, it achieves the alignment of lesion concepts with human cognition under the guidance of a retinal foundation model. Furthermore, to attain interpretability without losing lesion-specific information, our method employs a classifier built on a cross-attention mechanism for disease diagnosis and explanation, where explanations are grounded in the contributions of human-understandable lesion concepts and their visual localization. Notably, due to the structure and inherent interpretability of our model, clinicians can implement concept-level interventions to correct the diagnostic errors by simply adjusting erroneous lesion predictions. Experiments conducted on four fundus image datasets demonstrate that our method achieves favorable performance against state-of-the-art methods while providing faithful explanations and enabling conceptlevel interventions. Our code is publicly available at https://github.com/Sorades/CLAT.Keywords
Funding Information
- National Natural Science Foundation of China (62361166629)
- Key Research and Development Project of Hubei Province (2022BAD175, 2022BCA009)
- Interdisciplinary Innovative Talents Foundation from Renmin Hospital of Wuhan University
This publication has 40 references indexed in Scilit:
- A Foundation Language-Image Model of the Retina (FLAIR): encoding expert knowledge in text supervisionMedical Image Analysis, 2024
- Lesion-Aware Contrastive Learning for Diabetic Retinopathy Diagnosis2023
- ExAID: A multimodal explanation framework for computer-aided diagnosis of skin lesionsComputer Methods and Programs in Biomedicine, 2022
- Lesion-Based Contrastive Learning for Diabetic Retinopathy Grading from Fundus Images2021
- MIL-VT: Multiple Instance Learning Enhanced Vision Transformer for Fundus Image Classification2021
- Diagnostic assessment of deep learning algorithms for diabetic retinopathy screeningInformation Sciences, 2019
- Efficacy of a Deep Learning System for Detecting Glaucomatous Optic Neuropathy Based on Color Fundus PhotographsOphthalmology, 2018
- Zoom-in-Net: Deep Mining Lesions for Diabetic Retinopathy Detection2017
- Automated Identification of Diabetic Retinopathy Using Deep LearningOphthalmology, 2017
- Development and Validation of a Deep Learning Algorithm for Detection of Diabetic Retinopathy in Retinal Fundus PhotographsJAMA, 2016