Development and Validation of a Dynamic Real-Time Risk Prediction Model for Intensive Care Units Patients Based on Longitudinal Irregular Data: Multicenter Retrospective Study
Top Cited Papers
Open Access
- 23 April 2025
- journal article
- research article
- Published by JMIR Publications Inc. in Journal of Medical Internet Research
- Vol. 27 (1) , e69293
Abstract
Background: Timely and accurate prediction of short-term mortality is critical in intensive care units (ICUs), where patients’ conditions change rapidly. Traditional scoring systems, such as the Simplified Acute Physiology Score and Acute Physiology and Chronic Health Evaluation, rely on static variables collected within the first 24 hours of admission and do not account for continuously evolving clinical states. These systems lack real-time adaptability, interpretability, and generalizability. With the increasing availability of high-frequency electronic medical record (EMR) data, machine learning (ML) approaches have emerged as powerful tools to model complex temporal patterns and support dynamic clinical decision-making. However, existing models are often limited by their inability to handle irregular sampling and missing values, and many lack rigorous external validation across institutions. Objective: We aimed to develop a real-time, interpretable risk prediction model that continuously assesses ICU patient mortality using irregular, longitudinal EMR data, with improved performance and generalizability over traditional static scoring systems. Methods: A time-aware bidirectional attention-based long short-term memory (TBAL) model was developed using EMR data from the MIMIC-IV (Medical Information Mart for Intensive Care) and eICU Collaborative Research Database (eICU-CRD) databases, comprising 176,344 ICU stays. The model incorporated dynamic variables, including vital signs, laboratory results, and medication data, updated hourly, to perform static and continuous mortality risk assessments. External cross-validation and subgroup sensitivity analyses were conducted to evaluate robustness and fairness. Model performance was assessed using the area under the receiver operating characteristic curve (AUROC), area under the precision-recall curve (AUPRC), accuracy, and F1-score. Interpretability was enhanced using integrated gradients to identify key predictors. Results: For the static 12-hour to 1-day mortality prediction task, the TBAL model achieved AUROCs of 95.9 (95% CI 94.2-97.5) and 93.3 (95% CI 91.5-95.3) and AUPRCs of 48.5 and 21.6 in MIMIC-IV and eICU-CRD, respectively. Accuracy and F1-scores reached 94.1 and 46.7 in MIMIC-IV and 92.2 and 28.1 in eICU-CRD. In dynamic prediction tasks, AUROCs reached 93.6 (95% CI 93.2-93.9) and 91.9 (95% CI 91.6-92.1), with AUPRCs of 41.3 and 50, respectively. The model maintained high recall for positive cases (82.6% and 79.1% in MIMIC-IV and eICU-CRD). Cross-database validation yielded AUROCs of 81.3 and 76.1, confirming generalizability. Subgroup analysis showed stable performance across age, sex, and severity strata, with top predictors including lactate, vasopressor use, and Glasgow Coma Scale score. Conclusions: The TBAL model offers a robust, interpretable, and generalizable solution for dynamic real-time mortality risk prediction in ICU patients. Its ability to adapt to irregular temporal patterns and to provide hourly updated predictions positions it as a promising decision-support tool. Future work should validate its utility in prospective clinical trials and investigate its integration into real-world ICU workflows to enhance patient outcomes.Keywords
All Related Versions
This publication has 48 references indexed in Scilit:
- Real-time prediction of organ failures in patients with acute pancreatitis using longitudinal irregular dataJournal of Biomedical Informatics, 2023
- Benchmarking deep learning models on large healthcare datasetsJournal of Biomedical Informatics, 2018
- Mortality in the ICU: Who Dies Within the First 24 Hours?Chest, 2016
- Quality of patient care in the critical care unit in relation to nurse patient ratio: A descriptive studyIntensive and Critical Care Nursing, 2016
- Monitoring Technologies in Acute Care Environments2014
- Migrant and minority family members in the intensive care unit. A review of the literatureHeilberufeScience, 2013
- SAPS 3—From evaluation of the patient to evaluation of the intensive care unit. Part 2: Development of a prognostic model for hospital mortality at ICU admissionIntensive Care Medicine, 2005
- SAPS 3—From evaluation of the patient to evaluation of the intensive care unit. Part 1: Objectives, methods and cohort descriptionIntensive Care Medicine, 2005
- A new Simplified Acute Physiology Score (SAPS II) based on a European/North American multicenter studyJAMA, 1993
- Mortality Probability Models (MPM II) Based on an International Cohort of Intensive Care Unit PatientsJAMA, 1993