Automated sepsis prediction from unstructured electronic health records using natural language processing: a retrospective cohort study
Open Access
- 14 September 2025
- journal article
- Published by BMJ in BMJ Health & Care Informatics
- Vol. 32 (1) , e101354
Abstract
Objective: Artificial intelligence (AI) holds promise for predicting sepsis. However, challenges remain in integrating AI, natural language processing (NLP) and free text data to enhance sepsis diagnosis at emergency department (ED) triage. This study aimed to evaluate the effectiveness of AI in improving sepsis diagnosis. Methods: This retrospective cohort study analysed data from 134 266 patients admitted to the ED and subsequently hospitalised between 1 January 2016 and 31 December 2021. The data set comprised 10 variables and free-text triage comments, which underwent tokenisation and processing using a bag-of-words model. We evaluated four traditional NLP classifier models, including logistic regression, LightGBM, random forest and neural network. We also evaluated the performance of the BERT classifier. We used area under precision-recall curve (AUPRC) and area under the curve (AUC) as performance metrics. Results: Random forest exhibited superior predictive performance with an AUPRC of 0.789 (95% CI: 0.7668 to 0.8018) and an AUC of 0.80 (95% CI: 0.7842 to 0.8173). Using raw text, the BERT model achieved an AUPRC of 0.7542 (95% CI: 0.7418 to 0.7741) and AUC of 0.7735 (95% CI: 0.7628 to 0.8017) for sepsis prediction. Key variables included ED treatment time, patient age, arrival-to-treatment time, Australasian Triage Scale and visit type. Discussion: This study demonstrates AI, particularly random forest and BERT classifiers, for early sepsis detection in EDs using free-text patient concerns. Conclusion: Incorporating free text into machine learning improved diagnosis and identified missed cases, enhancing sepsis prediction in the ED with an AI-powered clinical decision support system. Large, prospective studies are needed to validate these findings.Keywords
This publication has 29 references indexed in Scilit:
- From Data to Optimal Decision Making: A Data-Driven, Probabilistic Machine Learning Approach to Decision Support for Patients With SepsisJMIR Medical Informatics, 2015
- Identification of the severe sepsis patient at triage: a prospective analysis of the Australasian Triage ScaleEmergency Medicine Journal, 2014
- Empiric Antibiotic Treatment Reduces Mortality in Severe Sepsis and Septic Shock From the First HourCritical Care Medicine, 2014
- Medical decision support using machine learning for early detection of late-onset neonatal sepsisJournal of the American Medical Informatics Association, 2014
- Emergency Department Abnormal Vital Sign “Triggers” Program Improves Time to TherapyAcademic Emergency Medicine, 2011
- Implementation of a real-time computerized sepsis alert in nonintensive care unit patients*Critical Care Medicine, 2011
- The role of nurses in the recognition and treatment of patients with sepsis in the emergency department: A prospective before-and-after intervention studyInternational Journal of Nursing Studies, 2010
- Long-term Cognitive Impairment and Functional Disability Among Survivors of Severe SepsisJAMA, 2010
- Regularization Paths for Generalized Linear Models via Coordinate Descent2010
- Epidemiology and economic evaluation of severe sepsis in France: age, severity, infection site, and place of acquisition (community, hospital, or intensive care unit) as determinants of workload and costJournal of Critical Care, 2005