Identifying future high healthcare utilization in patients with multimorbidity – development and internal validation of machine learning prediction models using electronic health record data
Open Access
- 13 February 2024
- journal article
- research article
- Published by Springer Nature in Health and Technology
- Vol. 14 (3) , 433-449
Abstract
Purpose: To develop and internally validate prediction models with machine learning for future potentially preventable healthcare utilization in patients with multiple long term conditions (MLTC). This study is the first step in investigating whether prediction models can help identify patients with MLTC that are most in need of integrated care. Methods: A retrospective cohort study was performed with electronic health record data from adults with MLTC from an academic medical center in the Netherlands. Based on demographic and healthcare utilization characteristics in 2017, we predicted ≥ 12 outpatient visits, ≥ 1 emergency department (ED) visits, and ≥ 1 acute hospitalizations in 2018. Four machine learning models (elastic net regression, extreme gradient boosting (XGB), logistic regression, and random forest) were developed, optimized, and evaluated in a hold-out dataset for each outcome. Results: A total of 14,486 patients with MLTC were included. Based on the area under the curve (AUC) and calibration curves, the XGB model was selected as final model for all three outcomes. The AUC was 0.82 for ≥ 12 outpatient visits, 0.76 for ≥ 1 ED visits and 0.73 for ≥ 1 acute hospitalizations. Despite adequate AUC and calibration, precision-recall curves showed suboptimal performance. Conclusions: The final selected models per outcome can identify patients with future potentially preventable high healthcare utilization. However, identifying high-risk patients with MLTC and substantiating if they are most in need of integrated care remains challenging. Further research is warranted investigating whether patients with high healthcare utilization are indeed the most in need of integrated care and whether quantitively identified patients match the identification based on clinicians’ experience and judgment.Keywords
This publication has 53 references indexed in Scilit:
- Aiding the prescriber: developing a machine learning approach to personalized risk modeling for chronic opioid therapy amongst US Army soldiersHealth Care Management Science, 2022
- Prediction Models for Future High-Need High-Cost Healthcare Use: a Systematic ReviewJournal of General Internal Medicine, 2022
- Examining variation in the measurement of multimorbidity in research: a systematic review of 566 studiesThe Lancet Public Health, 2021
- The impact of the COVID-19 pandemic on health services utilization in China: Time-series analyses for 2016–2020The Lancet Regional Health - Western Pacific, 2021
- “Hopes and wishes”: Goals of high-need, high-cost older patients and their caregiversPatient Education and Counseling, 2020
- Clinical Prediction Models2019
- Learning from Imbalanced Data Sets2018
- Evidence on multimorbidity from definition to intervention: An overview of systematic reviewsAgeing Research Reviews, 2017
- Current Challenges to Academic Health CentersJAMA, 2013
- Aging with multimorbidity: A systematic review of the literatureAgeing Research Reviews, 2011