Deciphering Diagnoses: How Large Language Models Explanations Influence Clinical Decision Making
Preprint
Abstract
Clinical Decision Support Systems (CDSS) utilize evidence-based knowledge and patient data to offer real-time recommendations, with Large Language Models (LLMs) emerging as a promising tool to generate plain-text explanations for medical decisions. This study explores the effectiveness and reliability of LLMs in generating explanations for diagnoses based on patient complaints. Three experienced doctors evaluated LLM-generated explanations of the connection between patient complaints and doctor and model-assigned diagnoses across several stages. Experimental results demonstrated that LLM explanations significantly increased doctors' agreement rates with given diagnoses and highlighted potential errors in LLM outputs, ranging from 5% to 30%. The study underscores the potential and challenges of LLMs in healthcare and emphasizes the need for careful integration and evaluation to ensure patient safety and optimal clinical utility.Keywords
All Related Versions
This publication has 10 references indexed in Scilit:
- Interpreting Black-Box Models: A Review on Explainable Artificial IntelligenceCognitive Computation, 2023
- Benefits, Limits, and Risks of GPT-4 as an AI Chatbot for MedicineNew England Journal of Medicine, 2023
- Medical image captioning via generative pretrained transformersScientific Reports, 2023
- ExplainCPE: A Free-text Explanation Benchmark of Chinese Pharmacist Examination2023
- SummQA at MEDIQA-Chat 2023: In-Context Learning with GPT-4 for Medical Summarization2023
- The Role of XAI in Advice-Taking from a Clinical Decision Support System: A Comparative User Study of Feature Contribution-Based and Example-Based ExplanationsApplied Sciences, 2022
- RuMedBench: A Russian Medical Language Understanding Benchmark2022
- Current Challenges and Future Opportunities for XAI in Machine Learning-Based Clinical Decision Support Systems: A Systematic ReviewApplied Sciences, 2021
- Explainability for artificial intelligence in healthcare: a multidisciplinary perspectiveBMC Medical Informatics and Decision Making, 2020
- Ten Commandments for Effective Clinical Decision Support: Making the Practice of Evidence-based Medicine a RealityJournal of the American Medical Informatics Association, 2003