Effects of explainable artificial intelligence in neurology decision support
Open Access
- 5 April 2024
- journal article
- research article
- Published by Wiley in Annals of Clinical and Translational Neurology
- Vol. 11 (5) , 1224-1235
Abstract
Objective: Artificial intelligence (AI)‐based decision support systems (DSS) are utilized in medicine but underlying decision‐making processes are usually unknown. Explainable AI (xAI) techniques provide insight into DSS, but little is known on how to design xAI for clinicians. Here we investigate the impact of various xAI techniques on a clinician's interaction with an AI‐based DSS in decision‐making tasks as compared to a general population.Methods: We conducted a randomized, blinded study in which members of the Child Neurology Society and American Academy of Neurology were compared to a general population. Participants received recommendations from a DSS via a random assignment of an xAI intervention (decision tree, crowd sourced agreement, case‐based reasoning, probability scores, counterfactual reasoning, feature importance, templated language, and no explanations). Primary outcomes included test performance and perceived explainability, trust, and social competence of the DSS. Secondary outcomes included compliance, understandability, and agreement per question.Results: We had 81 neurology participants with 284 in the general population. Decision trees were perceived as the more explainable by the medical versus general population (P < 0.01) and as more explainable than probability scores within the medical population (P < 0.001). Increasing neurology experience and perceived explainability degraded performance (P = 0.0214). Performance was not predicted by xAI method but by perceived explainability.Interpretation: xAI methods have different impacts on a medical versus general population; thus, xAI is not uniformly beneficial, and there is no one‐size‐fits‐all approach. Further user‐centered xAI research targeting clinicians and to develop personalized DSS for clinicians is needed.Keywords
Funding Information
- National Institutes of Health (KL2TR002381, UL1TR002378)
This publication has 32 references indexed in Scilit:
- Review of Machine Learning and Artificial Intelligence (ML/AI) for the Pediatric NeurologistPediatric Neurology, 2023
- Explainable artificial intelligence (XAI) in deep learning-based medical image analysisMedical Image Analysis, 2022
- The effects of explainability and causability on perception, trust, and acceptance: Implications for explainable AIInternational Journal of Human-Computer Studies, 2021
- Interpretable confidence measures for decision support systemsInternational Journal of Human-Computer Studies, 2020
- Real-Time Estimation of Drivers’ Trust in Automated Driving SystemsInternational Journal of Social Robotics, 2020
- Measuring the Quality of Explanations: The System Causability Scale (SCS)KI - Künstliche Intelligenz, 2020
- Causability and explainability of artificial intelligence in medicineWIREs Data Mining and Knowledge Discovery, 2019
- A naturalistic decision making perspective on studying intuitive decision making.Journal of Applied Research in Memory and Cognition, 2015
- Visual Interpretation of Kernel‐Based Prediction ModelsMolecular Informatics, 2011
- Taking stock of naturalistic decision makingJournal of Behavioral Decision Making, 2001