Large Language Model–Enhanced Electronic Frailty Assessment and Prognostic Performance in Hospitalized Adults: A Retrospective Cohort Study

Authors:
Preprint
Publisher website
Google Scholar
Abstract
Background: Electronic frailty indices (eFIs) derived from structured electronic health record (EHR) data may incompletely capture functional deficits. Large language models (LLMs) can extract frailty-related information from clinical notes, but whether improved frailty ascertainment improves prognostic performance is uncertain. Objective: To compare frailty ascertainment and prognostic performance between standard and LLM-enhanced eFIs among hospitalized adults. Design: Retrospective cohort study Participants: Adults aged ≥50 years enrolled in two studies at an academic medical center and affiliated community hospital between February 2022 and April 2024. Main Measures: A validated 35-deficit diagnosis-based eFI and an LLM-enhanced eFI incorporating deficits extracted from physical therapy (PT) documentation using GPT-4o. Frailty was defined using an index threshold of ≥0.20. The primary outcome was a composite of discharge to a facility, readmission, or death within 90 days. Multivariable logistic regression models assessed associations with the outcome, and discrimination was compared using area under the receiver operating characteristic curve (AUROC). Secondary analyses examined threshold-based classification and PT-documentation availability; prespecified sensitivity analyses evaluated population and outcome restrictions. Results: Among 616 hospitalizations, mean (SD) age was 70.9 (10.2) years, 329 (53.4%) had PT documentation, and 320 (51.9%) experienced the composite outcome. Standard and LLM-enhanced eFIs classified 216 (35.1%) and 362 (58.8%) hospitalizations as frail, respectively. Both were independently associated with the outcome (standard eFI: adjusted OR, 1.98 [95% CI, 1.39–2.83]; LLM-enhanced eFI: adjusted OR, 2.46 [95% CI, 1.75–3.47]). Discrimination did not differ significantly (AUROC, 0.636 vs 0.660; difference, 0.024 [95% CI, −0.006 to 0.053]; p =0.12). LLM enhancement increased sensitivity (42.5% to 69.4%) but decreased specificity (73.0% to 52.7%). Conclusions: LLM enhancement substantially increased frailty ascertainment and sensitivity for adverse outcomes but did not significantly improve prognostic discrimination. Thus, LLM-enhanced frailty assessment may depend on whether the intended use prioritizes broad identification of functional vulnerability or prediction of adverse post-hospital outcomes.

This publication has 48 references indexed in Scilit: