Validation of an Automatic Tagging System for Identifying Respiratory and Hemodynamic Deterioration Events in the Intensive Care Unit
Open Access
- 31 July 2021
- journal article
- research article
- Published by The Korean Society of Medical Informatics in Healthcare Informatics Research
- Vol. 27 (3) , 241-248
Abstract
Objectives: Predictive models for critical events in the intensive care unit (ICU) might help providers anticipate patient deterioration. At the heart of predictive model development lies the ability to accurately label significant events, thereby facilitating the use of machine learning and similar strategies. We conducted this study to establish the validity of an automated system for tagging respiratory and hemodynamic deterioration by comparing automatic tags to tagging by expert reviewers.Methods: This retrospective cohort study included 72,650 unique patient stays collected from Electronic Medical Records of the University of Massachusetts’ eICU. An enriched subgroup of stays was manually tagged by expert reviewers. The tags generated by the reviewers were compared to those generated by an automated system.Results: The automated system was able to rapidly and efficiently tag the complete database utilizing available clinical data. The overall agreement rate between the automated system and the clinicians for respiratory and hemodynamic deterioration tags was 89.4% and 87.1%, respectively. The automatic system did not add substantial variability beyond that seen among the reviewers.Conclusions: We demonstrated that a simple rule-based tagging system could provide a rapid and accurate tool for mass tagging of a compound database. These types of tagging systems may replace human reviewers and save considerable resources when trying to create a validated, labeled database used to train artificial intelligence algorithms. The ability to harness the power of artificial intelligence depends on efficient clinical validation of targeted conditions; hence, these systems and the methodology used to validate them are crucial.Keywords
This publication has 23 references indexed in Scilit:
- Signatures of Subacute Potentially Catastrophic Illness in the ICU: Model Development and Validation*Critical Care Medicine, 2016
- Early recognition of acutely deteriorating patients in non‐intensive care units: Assessment of an innovative monitoring technologyJournal of Hospital Medicine, 2012
- Independent Predictors and Outcomes of Unanticipated Early Postoperative Tracheal Intubation after Nonemergent, Noncardiac SurgeryAnesthesiology, 2011
- Early Recognition Of Acutely Deteriorating Patients In Non- Intensive Care Unit Setting: Assessment Of An Innovative Monitoring Technology2011
- Temporal Trends in Rates of Patient Harm Resulting from Medical CareNew England Journal of Medicine, 2010
- Heuristics to determine ventilation times of ICU patients from the MIMIC-II database2010
- Ten reasons why we should NOT use severity scores as entry criteria for clinical trials or in our treatment decisions*Critical Care Medicine, 2010
- Outreach and Early Warning Systems (EWS) for the prevention of Intensive Care admission and death of critically ill adult patients on general hospital wardsCochrane Database of Systematic Reviews, 2007
- Scoring systems for the critically ill:use, misuse and abuseCanadian Journal of Anesthesia/Journal canadien d'anesthésie, 2006
- Anticipating events of in-hospital cardiac arrestEuropean Journal of Emergency Medicine, 2004