Ignore, Trust, or Negotiate: Understanding Clinician Acceptance of AI-Based Treatment Recommendations in Health Care
Top Cited Papers
Open Access
- 19 April 2023
- conference paper
- Published by Association for Computing Machinery (ACM)
- p. 1-18
Abstract
Artificial intelligence (AI) in healthcare has the potential to improve patient outcomes, but clinician acceptance remains a critical barrier. We developed a novel decision support interface that provides interpretable treatment recommendations for sepsis, a life-threatening condition in which decisional uncertainty is common, treatment practices vary widely, and poor outcomes can occur even with optimal decisions. This system formed the basis of a mixed-methods study in which 24 intensive care clinicians made AI-assisted decisions on real patient cases. We found that explanations generally increased confidence in the AI, but concordance with specific recommendations varied beyond the binary acceptance or rejection described in prior work. Although clinicians sometimes ignored or trusted the AI, they also often prioritized aspects of the recommendations to follow, reject, or delay in a process we term “negotiation.” These results reveal novel barriers to adoption of treatment-focused AI tools and suggest ways to better support differing clinician perspectives.Keywords
All Related Versions
Funding Information
- Center for Machine Learning and Health, School of Computer Science, Carnegie Mellon University
- National Institutes of Health (R35HL144804)
- National Science Foundation (DGE2140739)
This publication has 62 references indexed in Scilit:
- State Sepsis Mandates — A New Era for Regulation of Hospital QualityNew England Journal of Medicine, 2017
- How good is the evidence to support primary care practice?BMJ Evidence-Based Medicine, 2017
- XGBoost2016
- Developing a New Definition and Assessing New Clinical Criteria for Septic ShockJAMA, 2016
- The Role of Explanations on Trust and Reliance in Clinical Decision Support Systems2015
- Too much, too little, or just right? Ways explanations impact end users' mental models2013
- An Appraisal of the Evidence Underlying Performance Measures for Community-acquired PneumoniaAmerican Journal of Respiratory and Critical Care Medicine, 2011
- Sepsis Bundles and Compliance With Clinical GuidelinesJournal of Intensive Care Medicine, 2011
- Anatomy of a DecisionJAMA, 1990
- Variations in Physician Practice: The Role of UncertaintyHealth Affairs, 1984