To Recommend or Not to Recommend: Designing and Evaluating AI-Enabled Decision Support for Time-Critical Medical Events
Open Access
- 16 October 2025
- journal article
- Published by Association for Computing Machinery (ACM) in Proceedings of the ACM on Human-Computer Interaction
- Vol. 9 (7) , 1-33
Abstract
AI-enabled decision-support systems aim to help medical providers rapidly make decisions with limited information during medical emergencies. A critical challenge in developing these systems is supporting providers in interpreting the system output to make optimal treatment decisions. In this study, we designed and evaluated an AI-enabled decision-support system to aid providers in treating patients with traumatic injuries. We first conducted user research with physicians to identify and design information types and AI outputs for a decision-support display. We then conducted an online experiment with 35 medical providers from six health systems to evaluate two human-AI interaction strategies: (1) AI information synthesis and (2) AI information and recommendations. We found that providers were more likely to make correct decisions when AI information and recommendations were provided compared to receiving no AI support. We also identified two socio-technical barriers to providing AI recommendations during time-critical medical events: (1) an accuracy-time trade-off in providing recommendations and (2) polarizing perceptions of recommendations between providers. We discuss three implications for developing AI-enabled decision support used in time-critical events, contributing to the limited research on human-AI interaction in this context.Keywords
All Related Versions
Funding Information
- National Institutes of Health (R01LM011834-05)
- National Science Foundation (IS-2107391)
- National Science Foundation Graduation Research Fellowship (204177)
This publication has 68 references indexed in Scilit:
- Development and validation of a Bayesian network predicting neurosurgical intervention after injury in children and adolescentsJournal of Trauma and Acute Care Surgery, 2023
- Development and validation of a Bayesian belief network predicting the probability of blood transfusion after pediatric injuryJournal of Trauma and Acute Care Surgery, 2022
- The flaws of policies requiring human oversight of government algorithmsComputer Law & Security Review, 2022
- Association of Clinician Diagnostic Performance With Machine Learning–Based Decision Support SystemsJAMA Network Open, 2021
- The ethics of AI in health care: A mapping reviewSocial Science & Medicine, 2020
- Establishing a Regional Trauma Preventable/Potentially Preventable Death RateAnnals of Surgery, 2020
- Odds Ratios—Current Best Practice and UseJAMA, 2018
- Opportunities for improvement in the management of patients who die from haemorrhage after traumaBritish Journal of Surgery, 2013
- Grand challenges in clinical decision supportJournal of Biomedical Informatics, 2008
- The Importance of Cognitive Errors in Diagnosis and Strategies to Minimize ThemAcademic Medicine, 2003