Operationalization of Artificial Intelligence Applications in the Intensive Care Unit
Top Cited Papers
Open Access
- 1 July 2025
- journal article
- review article
- Published by American Medical Association (AMA) in JAMA Network Open
- Vol. 8 (7) , e2522866
Abstract
Importance: Artificial intelligence (AI) presents transformative opportunities to address the increasing challenges faced by health care systems globally. Particularly, in data-rich environments, such as intensive care units (ICUs), AI could assist in enhancing clinical decision-making, streamline workflows, and improve patient outcomes. Despite these promising applications, the practical implementation of AI in clinical settings remains limited.Objective: To systematically evaluate AI system operationalization in the ICU, focusing on the AI field’s progress over time, technical maturity, and risk of bias.Evidence Review: In this systematic review, 5 databases (Embase, MEDLINE ALL, Web of Science Core Collection, Cochrane Central Register of Controlled Trials, and Google Scholar) were searched for studies published from July 28, 2020, to June 10, 2024. Eligible studies evaluated AI applications designed for use within ICUs for adults (aged ≥16 years) and used data collected during ICU stays. Two reviewers independently screened titles and abstracts, with a third reviewer resolving disagreements. Data extraction included AI application aims, dataset origins, technology readiness level (TRL) categorization, and the use of reporting standards. Risk of bias was assessed using the PROBAST (Prediction Model Study Risk of Bias Assessment Tool).Findings: Of 17 401 screened records, 1263 studies met the inclusion criteria. A total of 936 studies (74% of all studies) were classified as TRL 4 or below, indicating early-stage development or initial validation. Among these, 447 (37%) used internal datasets, 562 (46%) used MIMIC (Medical Information Mart for Intensive Care) datasets (I-IV), and 78 (6%) used the open-source eICU Collaborative Research Database. External validation (TRL 5) was achieved by 24% of studies. Only 25 (2%) progressed to clinical integration (TRL≥6), with no studies reaching full implementation (TRL 9). Although approximately half of generative AI models reached a higher TRL (14 [47%] with TRL 5), none reached clinical integration. Additionally, only 207 studies (16%) referenced reporting standards, with adherence modestly increasing from 14% in 2021 to 23% in 2024. High risk of bias was identified in 581 of 1103 studies (53%), primarily due to methodologic shortcomings in the analysis domain.Conclusions and Relevance: Despite substantial growth in AI research within intensive care medicine in recent years, the transition from development to clinical implementation still remains limited and has made little progress over time. A paradigm shift is urgently required in the medical literature—one that moves beyond retrospective validation toward the operationalization and prospective testing of AI for tangible clinical impact.Keywords
This publication has 37 references indexed in Scilit:
- Moving Toward Implementation of Responsible Artificial Intelligence in Health CareJAMA, 2025
- Testing and Evaluation of Health Care Applications of Large Language ModelsJAMA, 2025
- A multinational study on artificial intelligence adoption: Clinical implementers' perspectivesInternational Journal of Medical Informatics, 2024
- A Nationwide Network of Health AI Assurance LaboratoriesJAMA, 2024
- AI in Medicine—JAMA’s Focus on Clinical Outcomes, Patient-Centered Care, Quality, and EquityJAMA, 2023
- Moving from bytes to bedside: a systematic review on the use of artificial intelligence in the intensive care unitIntensive Care Medicine, 2021
- Algorithmovigilance—Advancing Methods to Analyze and Monitor Artificial Intelligence–Driven Health Care for Effectiveness and EquityJAMA Network Open, 2021
- Comparison of Methods to Reduce Bias From Clinical Prediction Models of Postpartum DepressionJAMA Network Open, 2021
- Machine learning in intensive care medicine: ready for take-off?Intensive Care Medicine, 2020
- Living systematic review: 1. Introduction—the why, what, when, and howJournal of Clinical Epidemiology, 2017