External validation of a proprietary risk model for 1-year mortality in community-dwelling adults aged 65 years or older

Authors:

Abstract
Objective: To examine the discrimination, calibration, and algorithmic fairness of the Epic End of Life Care Index (EOL-CI). Materials and Methods: We assessed the EOL-CI’s performance by estimating area under the receiver operating characteristic curve (AUC), sensitivity, and positive and negative predictive values in community-dwelling adults ≥65 years of age in a single health system in the Southeastern United States. Algorithmic fairness was examined by comparing the model’s performance across sex, race, and ethnicity subgroups. Using a machine learning approach, we also explored local re-calibration of the EOL-CI considering additional information on past hospitalizations and frailty. Results: Among 215 731 patients (median age = 74 years, 57% female, 12% of Black race), 10% were classified as medium risk (15-44) and 3% as high risk (≥45) by the EOL-CI. The observed 1-year mortality rate was 3%. The EOL-CI had an AUC 0.82 for 1-year mortality, with a positive predictive value of 22%. Predictive performance was generally similar across sex and race subgroups, though the EOL-CI displayed better performance with increasing age and in older adults with 2 or more outpatient encounters in the past 24 months. Local re-calibration of the EOL-CI was required to provide absolute estimates of mortality risk, and calibration was further improved when the EOL-CI was augmented with data on inpatient hospitalizations and frailty. Discussion: The EOL-CI demonstrates reasonable discrimination, albeit with better performance in older adults and in those with greater health system contact. Conclusion: Local refinement and calibration of the EOL-CI score is required to provide direct estimates of prognosis, with the goal of making the EOL-CI a more a valuable tool at the point of care for identifying patients who would benefit from targeted palliative care interventions and proactive care planning.
Funding Information
  • National Institute on Aging
  • National Institutes of Health (U54AG063546)
  • Imbedded Pragmatic Alzheimer's Disease
  • AD-Related Dementias Clinical Trials Collaboratory
  • National Center for Advancing Translational Sciences
  • National Institutes of Health (UL1TR004929, K23AG070234, K23AG073529)

This publication has 48 references indexed in Scilit: