Clinical XAI: A Tutorial on Concepts, Methods, and Modalities from Linear Models to LLMs
Open Access
- 26 May 2026
- journal article
- Published by Association for Computing Machinery (ACM) in ACM Computing Surveys
- Vol. 58 (15) , 1-36
Abstract
Explainable AI (XAI) in clinical risk prediction helps make applied machine learning models transparent and clinically accountable. This tutorial covers recent progress across EHR, text, imaging, genomics, and multimodal data, with a later focus on emerging large language model (LLM) systems. We aimed to standardise terminology and distinguish terms such as interpretability, explanation, fairness, bias, trust, and transparency. We also provide a decision map linking methods to evaluation requirements. We review 97 unique studies (2019–2024) using a scheme that distinguishes alignment with knowledge (AK), expert review (ER), decision impact (DI), workflow integration (WI), and patient-facing (PF) assessment. We find that SHAP with tree ensembles is the most popular method in tabular EHRs, attention and CAM methods are common in imaging, and saliency-plus enrichment testing in genomics. A minority of papers report clinician-involved validation, and reporting of rater agreement, calibration, and subgroup effects is inconsistent. We, thus, propose a tiered, modality-agnostic framework, comprising AK+ER as the minimum standard, followed by DI (accuracy, time, confidence), and then WI (usage, outcomes), and provide design checklists, metrics, and pitfalls. Our aim is a roadmap, grounded in clinical practice, for developing, stress-testing, and deploying XAI in risk prediction.Keywords
This publication has 130 references indexed in Scilit:
- Dual explainability framework for heart disease prediction using LIME and permutation feature importanceDiscover Applied Sciences, 2025
- A review of deep learning models and online healthcare databases for electronic health records and their use for health predictionArtificial Intelligence Review, 2024
- Class imbalance on medical image classification: towards better evaluation practices for discrimination and calibration performanceEuropean Radiology, 2024
- Designing explainable AI to improve human-AI team performance: A medical stakeholder-driven scoping reviewArtificial Intelligence in Medicine, 2024
- DC-SHAP Method for Consistent Explainability in Privacy-Preserving Distributed Machine LearningHuman-Centric Intelligent Systems, 2023
- Explainable AI for clinical and remote health applications: a survey on tabular and time series dataArtificial Intelligence Review, 2022
- Deep learning in generating radiology reports: A surveyArtificial Intelligence in Medicine, 2020
- Potential Liability for Physicians Using Artificial IntelligenceJAMA, 2019
- Supersparse linear integer models for optimized medical scoring systemsMachine Learning, 2015
- The use of expert systems in the healthcare industryInformation & Management, 1992