An explainable transformer model integrating PET and tabular data for histologic grading and prognosis of follicular lymphoma: a multi-institutional digital biopsy study
- 30 January 2025
- journal article
- research article
- Published by Springer Nature in European Journal of Nuclear Medicine and Molecular Imaging
- Vol. 52 (7) , 2384-2396
Abstract
Background Pathological grade is a critical determinant of clinical outcomes and decision-making of follicular lymphoma (FL). This study aimed to develop a deep learning model as a digital biopsy for the non-invasive identification of FL grade. Methods This study retrospectively included 513 FL patients from five independent hospital centers, randomly divided into training, internal validation, and external validation cohorts. A multimodal fusion Transformer model was developed integrating 3D PET tumor images with tabular data to predict FL grade. Additionally, the model is equipped with explainable modules, including Gradient-weighted Class Activation Mapping (Grad-CAM) for PET images, SHapley Additive exPlanations analysis for tabular data, and the calculation of predictive contribution ratios for both modalities, to enhance clinical interpretability and reliability. The predictive performance was evaluated using the area under the receiver operating characteristic curve (AUC) and accuracy, and its prognostic value was also assessed. Results The Transformer model demonstrated high accuracy in grading FL, with AUCs of 0.964–0.985 and accuracies of 90.2-96.7% in the training cohort, and similar performance in the validation cohorts (AUCs: 0.936–0.971, accuracies: 86.4-97.0%). Ablation studies confirmed that the fusion model outperformed single-modality models (AUCs: 0.974 − 0.956, accuracies: 89.8%-85.8%). Interpretability analysis revealed that PET images contributed 81-89% of the predictive value. Grad-CAM highlighted the tumor and peri-tumor regions. The model also effectively stratified patients by survival risk (P < 0.05), highlighting its prognostic value. Conclusions Our study developed an explainable multimodal fusion Transformer model for accurate grading and prognosis of FL, with the potential to aid clinical decision-making. Graphical AbstractKeywords
Funding Information
- The National Natural Science Foundation of China (81971653)
- the 1·3·5 project for disciplines of excellence, West China Hospital, Sichuan University (ZYAI24014)
This publication has 33 references indexed in Scilit:
- Designing interpretable ML system to enhance trust in healthcare: A systematic review to proposed responsible clinician-AI-collaboration frameworkInformation Fusion, 2024
- Deep learning techniques in PET/CT imaging: A comprehensive review from sinogram to image spaceComputer Methods and Programs in Biomedicine, 2023
- Follicular lymphoma: 2023 update on diagnosis and managementAmerican Journal of Hematology, 2022
- Swin UNETR: Swin Transformers for Semantic Segmentation of Brain Tumors in MRI Images2022
- Explanation of machine learning models using shapley additive explanation and application for real data in hospitalComputer Methods and Programs in Biomedicine, 2022
- Machine learning in the differentiation of follicular lymphoma from diffuse large B-cell lymphoma with radiomic [18F]FDG PET/CT featuresEuropean Journal of Nuclear Medicine and Molecular Imaging, 2021
- Explaining individual predictions when features are dependent: More accurate approximations to Shapley valuesArtificial Intelligence, 2021
- Prognostic significance of histologic grade and Ki‐67 proliferation index in follicular lymphomaHematological Oncology, 2020
- Grad-CAM: Visual Explanations from Deep Networks via Gradient-Based LocalizationInternational Journal of Computer Vision, 2019
- Evaluating Novel PET-CT Functional Parameters TLG and TMTV in Differentiating Low-grade Versus Grade 3A Follicular LymphomaClinical Lymphoma Myeloma and Leukemia, 2019