Corey Chivers
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Previous Organizations / Institutes
The organizations / institutes from the scholar's other articles in the past ten years
Corey Chivers has a total of 70 co-authors. Here are the top co-authors by number of shared publications:
Justin E. Bekelman
University of Pennsylvania, Department of Radiation Oncology, Philadelphia, PA
3 shared publicationsLawrence N. Shulman
Abramson Cancer Center, University of Pennsylvania, Philadelphia, PA
3 shared publicationsMitesh S. Patel
University of Pennsylvania, Philadelphia, PA
3 shared publicationsAmol S. Navathe
University of Pennsylvania, Philadelphia, PA
2 shared publications- Anahita Davoudi
Center for Home Care Policy & Research, VNS Health New York New York USA
2 shared publications Andrew Crane-Droesch
University of Pennsylvania and Penn Medicine Predictive Healthcare, Philadelphia, Pennsylvania (A.C.)
2 shared publications
Latest Publications
- Open Access
- 1 December 2022
- Published by American Society of Clinical Oncology (ASCO) in JCO Clinical Cancer Informatics
- Vol. 6 (6) , e2200073
Development of Machine Learning Algorithms Incorporating Electronic Health Record Data, Patient-Reported Outcomes, or Both to Predict Mortality for Outpatients With Cancer
PURPOSE: Machine learning (ML) algorithms that incorporate routinely collected patient-reported outcomes (PROs) alongside electronic health record (EHR) variables may improve prediction of short-term mortality and facilitate earlier supportive and palliative care for patients with cancer.METHODS: We trained and validated two-phase ML algorithms that incorporated standard PRO assessments alongside approximately 200 routinely collected EHR variables, among patients with medical oncology encounters at a tertiary academic oncology and a community oncology practice.RESULTS: Among 12,350 patients, 5,870 (47.5%) completed PRO assessments. Compared with EHR- and PRO-only algorithms, the EHR + PRO model improved predictive performance in both tertiary oncology (EHR + PRO v EHR v PRO: area under the curve [AUC] 0.86 [0.85-0.87] v 0.82 [0.81-0.83] v 0.74 [0.74-0.74]) and community oncology (area under the curve 0.89 [0.88-0.90] v 0.86 [0.85-0.88] v 0.77 [0.76-0.79]) practices.CONCLUSION: Routinely collected PROs contain added prognostic information not captured by an EHR-based ML mortality risk algorithm. Augmenting an EHR-based algorithm with PROs resulted in a more accurate and clinically relevant model, which can facilitate earlier and targeted supportive care for patients with cancer.
- 1 December 2022
- Published by American Society of Clinical Oncology (ASCO) in JCO Clinical Cancer Informatics
- Vol. 6 (6) , e2200073
Open AccessDevelopment of Machine Learning Algorithms Incorporating Electronic Health Record Data, Patient-Reported Outcomes, or Both to Predict Mortality for Outpatients With Cancer
PURPOSE: Machine learning (ML) algorithms that incorporate routinely collected patient-reported outcomes (PROs) alongside electronic health record (EHR) variables may improve prediction of short-term mortality and facilitate earlier supportive and palliative care for patients with cancer.METHODS: We trained and validated two-phase ML algorithms that incorporated standard PRO assessments alongside approximately 200 routinely collected EHR variables, among patients with medical oncology encounters at a tertiary academic oncology and a community oncology practice.RESULTS: Among 12,350 patients, 5,870 (47.5%) completed PRO assessments. Compared with EHR- and PRO-only algorithms, the EHR + PRO model improved predictive performance in both tertiary oncology (EHR + PRO v EHR v PRO: area under the curve [AUC] 0.86 [0.85-0.87] v 0.82 [0.81-0.83] v 0.74 [0.74-0.74]) and community oncology (area under the curve 0.89 [0.88-0.90] v 0.86 [0.85-0.88] v 0.77 [0.76-0.79]) practices.CONCLUSION: Routinely collected PROs contain added prognostic information not captured by an EHR-based ML mortality risk algorithm. Augmenting an EHR-based algorithm with PROs resulted in a more accurate and clinically relevant model, which can facilitate earlier and targeted supportive care for patients with cancer.
- Open Access
- 21 November 2022
- Published by Oxford University Press (OUP) in Journal of the American Medical Informatics Association
- Vol. 30 (2) , 348-354
Performance drift in a mortality prediction algorithm among patients with cancer during the SARS-CoV-2 pandemic
Sudden changes in health care utilization during the severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) pandemic may have impacted the performance of clinical predictive models that were trained prior to the pandemic. In this study, we evaluated the performance over time of a machine learning, electronic health record-based mortality prediction algorithm currently used in clinical practice to identify patients with cancer who may benefit from early advance care planning conversations. We show that during the pandemic period, algorithm identification of high-risk patients had a substantial and sustained decline. Decreases in laboratory utilization during the peak of the pandemic may have contributed to drift. Calibration and overall discrimination did not markedly decline during the pandemic. This argues for careful attention to the performance and retraining of predictive algorithms that use inputs from the pandemic period.
- 21 November 2022
- Published by Oxford University Press (OUP) in Journal of the American Medical Informatics Association
- Vol. 30 (2) , 348-354
Open AccessPerformance drift in a mortality prediction algorithm among patients with cancer during the SARS-CoV-2 pandemic
Sudden changes in health care utilization during the severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) pandemic may have impacted the performance of clinical predictive models that were trained prior to the pandemic. In this study, we evaluated the performance over time of a machine learning, electronic health record-based mortality prediction algorithm currently used in clinical practice to identify patients with cancer who may benefit from early advance care planning conversations. We show that during the pandemic period, algorithm identification of high-risk patients had a substantial and sustained decline. Decreases in laboratory utilization during the peak of the pandemic may have contributed to drift. Calibration and overall discrimination did not markedly decline during the pandemic. This argues for careful attention to the performance and retraining of predictive algorithms that use inputs from the pandemic period.
- Open Access
- 3 December 2020
- Published by JMIR Publications Inc. in Journal of Medical Internet Research
- Vol. 22 (12) , e22493
Patient Interaction Phenotypes With an Automated Remote Hypertension Monitoring Program and Their Association With Blood Pressure Control: Observational Study
Background: Automated texting platforms have emerged as a tool to facilitate communication between patients and health care providers with variable effects on achieving target blood pressure (BP). Understanding differences in the way patients interact with these communication platforms can inform their use and design for hypertension management. Objective: Our primary aim was to explore the unique phenotypes of patient interactions with an automated text messaging platform for BP monitoring. Our secondary aim was to estimate associations between interaction phenotypes and BP control. Methods: This study was a secondary analysis of data from a randomized controlled trial for adults with poorly controlled hypertension. A total of 201 patients with established primary care were assigned to the automated texting platform; messages exchanged throughout the 4-month program were analyzed. We used the k-means clustering algorithm to characterize two different interaction phenotypes: program conformity and engagement style. First, we identified unique clusters signifying differences in program conformity based on the frequency over time of error alerts, which were generated to patients when they deviated from the requested text message format (eg, ###/## for BP). Second, we explored overall engagement styles, defined by error alerts and responsiveness to text prompts, unprompted messages, and word count averages. Finally, we applied the chi-square test to identify associations between each interaction phenotype and achieving the target BP. Results: We observed 3 categories of program conformity based on their frequency of error alerts: those who immediately and consistently submitted texts without system errors (perfect users, 51/201), those who did so after an initial learning period (adaptive users, 66/201), and those who consistently submitted messages generating errors to the platform (nonadaptive users, 38/201). Next, we observed 3 categories of engagement style: the enthusiast, who tended to submit unprompted messages with high word counts (17/155); the student, who inconsistently engaged (35/155); and the minimalist, who engaged only when prompted (103/155). Of all 6 phenotypes, we observed a statistically significant association between patients demonstrating the minimalist communication style (high adherence, few unprompted messages, limited information sharing) and achieving target BP (P<.001). Conclusions: We identified unique interaction phenotypes among patients engaging with an automated text message platform for remote BP monitoring. Only the minimalist communication style was associated with achieving target BP. Identifying and understanding interaction phenotypes may be useful for tailoring future automated texting interactions and designing future interventions to achieve better BP control.
- 3 December 2020
- Published by JMIR Publications Inc. in Journal of Medical Internet Research
- Vol. 22 (12) , e22493
Open AccessPatient Interaction Phenotypes With an Automated Remote Hypertension Monitoring Program and Their Association With Blood Pressure Control: Observational Study
Background: Automated texting platforms have emerged as a tool to facilitate communication between patients and health care providers with variable effects on achieving target blood pressure (BP). Understanding differences in the way patients interact with these communication platforms can inform their use and design for hypertension management. Objective: Our primary aim was to explore the unique phenotypes of patient interactions with an automated text messaging platform for BP monitoring. Our secondary aim was to estimate associations between interaction phenotypes and BP control. Methods: This study was a secondary analysis of data from a randomized controlled trial for adults with poorly controlled hypertension. A total of 201 patients with established primary care were assigned to the automated texting platform; messages exchanged throughout the 4-month program were analyzed. We used the k-means clustering algorithm to characterize two different interaction phenotypes: program conformity and engagement style. First, we identified unique clusters signifying differences in program conformity based on the frequency over time of error alerts, which were generated to patients when they deviated from the requested text message format (eg, ###/## for BP). Second, we explored overall engagement styles, defined by error alerts and responsiveness to text prompts, unprompted messages, and word count averages. Finally, we applied the chi-square test to identify associations between each interaction phenotype and achieving the target BP. Results: We observed 3 categories of program conformity based on their frequency of error alerts: those who immediately and consistently submitted texts without system errors (perfect users, 51/201), those who did so after an initial learning period (adaptive users, 66/201), and those who consistently submitted messages generating errors to the platform (nonadaptive users, 38/201). Next, we observed 3 categories of engagement style: the enthusiast, who tended to submit unprompted messages with high word counts (17/155); the student, who inconsistently engaged (35/155); and the minimalist, who engaged only when prompted (103/155). Of all 6 phenotypes, we observed a statistically significant association between patients demonstrating the minimalist communication style (high adherence, few unprompted messages, limited information sharing) and achieving target BP (P<.001). Conclusions: We identified unique interaction phenotypes among patients engaging with an automated text message platform for remote BP monitoring. Only the minimalist communication style was associated with achieving target BP. Identifying and understanding interaction phenotypes may be useful for tailoring future automated texting interactions and designing future interventions to achieve better BP control.
- Open Access
- 20 October 2020
- Published by American College of Physicians in Annals of Internal Medicine
- Vol. 173 (8) , 680-681
Locally Informed Simulation to Predict Hospital Capacity Needs During the COVID-19 Pandemic
- 20 October 2020
- Published by American College of Physicians in Annals of Internal Medicine
- Vol. 173 (8) , 680-681
Open AccessLocally Informed Simulation to Predict Hospital Capacity Needs During the COVID-19 Pandemic
- Open Access
- 17 July 2020
- Published by JMIR Publications Inc. in JMIR Preprints
Patient Interaction Phenotypes With an Automated Remote Hypertension Monitoring Program and Their Association With Blood Pressure Control: Observational Study (Preprint)
Journal of Medical Internet Research - International Scientific Journal for Medical Research, Information and Communication on the Internet #Preprint #PeerReviewMe: Warning: This is a unreviewed preprint. Readers are warned that the document has not been peer-reviewed by expert/patient reviewers or an academic editor, may contain misleading claims, and is likely to undergo changes before final publication, if accepted, or may have been rejected/withdrawn. Readers with interest and expertise are encouraged to sign up as peer-reviewer, if the paper is within an open peer-review period. Please cite this preprint only for review purposes or for grant applications and CVs (if you are the author). Background: Automated texting platforms have emerged as a tool to facilitate communication between patients and health care providers with variable effects on achieving target blood pressure (BP). Understanding differences in the way patients interact with these communication platforms can inform their use and design for hypertension management. Objective: Our primary aim was to explore the unique phenotypes of patient interactions with an automated text messaging platform for BP monitoring. Our secondary aim was to estimate associations between interaction phenotypes and BP control. Methods: This study was a secondary analysis of data from a randomized controlled trial for adults with poorly controlled hypertension. A total of 201 patients with established primary care were assigned to the automated texting platform; messages exchanged throughout the 4-month program were analyzed. We used the k-means clustering algorithm to characterize two different interaction phenotypes: program conformity and engagement style. First, we identified unique clusters signifying differences in program conformity based on the frequency over time of error alerts, which were generated to patients when they deviated from the requested text message format (eg, ###/## for BP). Second, we explored overall engagement styles, defined by error alerts and responsiveness to text prompts, unprompted messages, and word count averages. Finally, we applied the chi-square test to identify associations between each interaction phenotype and achieving the target BP. Results: We observed 3 categories of program conformity based on their frequency of error alerts: those who immediately and consistently submitted texts without system errors (perfect users, 51/201), those who did so after an initial learning period (adaptive users, 66/201), and those who consistently submitted messages generating errors to the platform (nonadaptive users, 38/201). Next, we observed 3 categories of engagement style: the enthusiast, who tended to submit unprompted messages with high word counts (17/155); the student, who inconsistently engaged (35/155); and the minimalist, who engaged only when prompted (103/155). Of all 6 phenotypes, we observed a statistically significant association between patients demonstrating the minimalist communication style (high adherence, few unprompted messages, limited information sharing) and achieving target BP (P<.001). Conclusions: We identified unique interaction phenotypes among patients engaging with an automated text message platform for remote BP monitoring. Only the minimalist communication style was associated with achieving target BP. Identifying and understanding interaction phenotypes may be useful for tailoring future automated texting interactions and designing future interventions to achieve better BP control.
- 17 July 2020
- Published by JMIR Publications Inc. in JMIR Preprints
Open AccessPatient Interaction Phenotypes With an Automated Remote Hypertension Monitoring Program and Their Association With Blood Pressure Control: Observational Study (Preprint)
Journal of Medical Internet Research - International Scientific Journal for Medical Research, Information and Communication on the Internet #Preprint #PeerReviewMe: Warning: This is a unreviewed preprint. Readers are warned that the document has not been peer-reviewed by expert/patient reviewers or an academic editor, may contain misleading claims, and is likely to undergo changes before final publication, if accepted, or may have been rejected/withdrawn. Readers with interest and expertise are encouraged to sign up as peer-reviewer, if the paper is within an open peer-review period. Please cite this preprint only for review purposes or for grant applications and CVs (if you are the author). Background: Automated texting platforms have emerged as a tool to facilitate communication between patients and health care providers with variable effects on achieving target blood pressure (BP). Understanding differences in the way patients interact with these communication platforms can inform their use and design for hypertension management. Objective: Our primary aim was to explore the unique phenotypes of patient interactions with an automated text messaging platform for BP monitoring. Our secondary aim was to estimate associations between interaction phenotypes and BP control. Methods: This study was a secondary analysis of data from a randomized controlled trial for adults with poorly controlled hypertension. A total of 201 patients with established primary care were assigned to the automated texting platform; messages exchanged throughout the 4-month program were analyzed. We used the k-means clustering algorithm to characterize two different interaction phenotypes: program conformity and engagement style. First, we identified unique clusters signifying differences in program conformity based on the frequency over time of error alerts, which were generated to patients when they deviated from the requested text message format (eg, ###/## for BP). Second, we explored overall engagement styles, defined by error alerts and responsiveness to text prompts, unprompted messages, and word count averages. Finally, we applied the chi-square test to identify associations between each interaction phenotype and achieving the target BP. Results: We observed 3 categories of program conformity based on their frequency of error alerts: those who immediately and consistently submitted texts without system errors (perfect users, 51/201), those who did so after an initial learning period (adaptive users, 66/201), and those who consistently submitted messages generating errors to the platform (nonadaptive users, 38/201). Next, we observed 3 categories of engagement style: the enthusiast, who tended to submit unprompted messages with high word counts (17/155); the student, who inconsistently engaged (35/155); and the minimalist, who engaged only when prompted (103/155). Of all 6 phenotypes, we observed a statistically significant association between patients demonstrating the minimalist communication style (high adherence, few unprompted messages, limited information sharing) and achieving target BP (P<.001). Conclusions: We identified unique interaction phenotypes among patients engaging with an automated text message platform for remote BP monitoring. Only the minimalist communication style was associated with achieving target BP. Identifying and understanding interaction phenotypes may be useful for tailoring future automated texting interactions and designing future interventions to achieve better BP control.
- Top Cited Papers
- 7 July 2020
- Published by American College of Physicians in Annals of Internal Medicine
- Vol. 173 (1) , 21-28
Locally Informed Simulation to Predict Hospital Capacity Needs During the COVID-19 Pandemic
The coronavirus disease 2019 (COVID-19) pandemic challenges hospital leaders to make time-sensitive, critical decisions about clinical operations and resource allocations. To estimate the timing of surges in clinical demand and the best- and worst-case scenarios of local COVID-19–induced strain on hospital capacity, and thus inform clinical operations and staffing demands and identify when hospital capacity would be saturated. Monte Carlo simulation instantiation of a susceptible, infected, removed (SIR) model with a 1-day cycle. 3 hospitals in an academic health system. All people living in the greater Philadelphia region. The COVID-19 Hospital Impact Model (CHIME) (http://penn-chime.phl.io) SIR model was used to estimate the time from 23 March 2020 until hospital capacity would probably be exceeded, and the intensity of the surge, including for intensive care unit (ICU) beds and ventilators. Using patients with COVID-19 alone, CHIME estimated that it would be 31 to 53 days before demand exceeds existing hospital capacity. In best- and worst-case scenarios of surges in the number of patients with COVID-19, the needed total capacity for hospital beds would reach 3131 to 12 650 across the 3 hospitals, including 338 to 1608 ICU beds and 118 to 599 ventilators. Model parameters were taken directly or derived from published data across heterogeneous populations and practice environments and from the health system's historical data. CHIME does not incorporate more transition states to model infection severity, social networks to model transmission dynamics, or geographic information to account for spatial patterns of human interaction. Publicly available and designed for hospital operations leaders, this modeling tool can inform preparations for capacity strain during the early days of a pandemic. University of Pennsylvania Health System and the Palliative and Advanced Illness Research Center.
- 7 July 2020
- Published by American College of Physicians in Annals of Internal Medicine
- Vol. 173 (1) , 21-28
Top Cited PapersLocally Informed Simulation to Predict Hospital Capacity Needs During the COVID-19 Pandemic
The coronavirus disease 2019 (COVID-19) pandemic challenges hospital leaders to make time-sensitive, critical decisions about clinical operations and resource allocations. To estimate the timing of surges in clinical demand and the best- and worst-case scenarios of local COVID-19–induced strain on hospital capacity, and thus inform clinical operations and staffing demands and identify when hospital capacity would be saturated. Monte Carlo simulation instantiation of a susceptible, infected, removed (SIR) model with a 1-day cycle. 3 hospitals in an academic health system. All people living in the greater Philadelphia region. The COVID-19 Hospital Impact Model (CHIME) (http://penn-chime.phl.io) SIR model was used to estimate the time from 23 March 2020 until hospital capacity would probably be exceeded, and the intensity of the surge, including for intensive care unit (ICU) beds and ventilators. Using patients with COVID-19 alone, CHIME estimated that it would be 31 to 53 days before demand exceeds existing hospital capacity. In best- and worst-case scenarios of surges in the number of patients with COVID-19, the needed total capacity for hospital beds would reach 3131 to 12 650 across the 3 hospitals, including 338 to 1608 ICU beds and 118 to 599 ventilators. Model parameters were taken directly or derived from published data across heterogeneous populations and practice environments and from the health system's historical data. CHIME does not incorporate more transition states to model infection severity, social networks to model transmission dynamics, or geographic information to account for spatial patterns of human interaction. Publicly available and designed for hospital operations leaders, this modeling tool can inform preparations for capacity strain during the early days of a pandemic. University of Pennsylvania Health System and the Palliative and Advanced Illness Research Center.
- 20 May 2020
- Published by American Society of Clinical Oncology (ASCO) in Journal of Clinical Oncology
- Vol. 38 (15_suppl) , 2009
Prospective validation of a machine learning algorithm to predict short-term mortality among outpatients with cancer.
2009 Background: Oncologists accurately identify only 35% of patients with cancer who will die in six months. There is an urgent need for automated, accurate prognostic systems to inform treatment and advance care planning in oncology. We assessed the prospective performance of a previously described ML algorithm (Parikh et al, JAMA Netw Open, 2019) to predict short-term mortality in a cohort of general oncology outpatients. Methods: Our prospective cohort consisted of patients aged ≥18 years who had a medical or gynecologic oncology encounter between March 1 and April 30, 2019 in either a tertiary academic practice or one of twelve community practices within a large academic cancer system. We used a retrospectively validated gradient-boosting ML algorithm, based on 559 structured electronic health record (EHR) variables, to predict 180-day mortality prior to each oncology encounter. For patients with multiple encounters, we selected the last encounter to assess performance. We assessed several performance metrics, including area under the receiver operating curve (AUC), area under the precision-recall curve (AUPRC), scaled Brier score (sBrier; a measure of calibration ranging from 0 [random] to 1 [perfect]), and positive predictive value (PPV). Results: Of 25,537 unique patients, median age was 64.4 (interquartile range 53.3 – 73.0), 76.8% were White, 56.5% were treated at a community center, and 4.1% died within 180 days. The ML algorithm had an AUC of 0.89 (95% confidence interval [CI] 0.88-0.90), AUPRC 0.34, and sBrier 0.29. At a prespecified threshold of 40%, observed 180-day mortality was 44.5% (95% CI 40.7 – 48.4%) in the high-risk group vs. 3.0% (95% CI 2.8% – 3.3%) in the low-risk group. There was an 85-fold difference in mortality (13.6% vs. 0.16%) in the top vs. bottom risk quartiles. The model was well-calibrated for mortality risks ≤40% and slightly under-calibrated for mortality risks > 40%. Performance varied across cancer types in the tertiary hospital but did not vary by race or practice type (Table). Conclusions: In this prospective cohort study among outpatients with cancer, a ML prognostic algorithm based on EHR data had better discrimination and calibration that published cancer-specific models. This is one of the first ML prognostic models to be prospectively validated in oncology. [Table: see text]
- 20 May 2020
- Published by American Society of Clinical Oncology (ASCO) in Journal of Clinical Oncology
- Vol. 38 (15_suppl) , 2009
Prospective validation of a machine learning algorithm to predict short-term mortality among outpatients with cancer.
2009 Background: Oncologists accurately identify only 35% of patients with cancer who will die in six months. There is an urgent need for automated, accurate prognostic systems to inform treatment and advance care planning in oncology. We assessed the prospective performance of a previously described ML algorithm (Parikh et al, JAMA Netw Open, 2019) to predict short-term mortality in a cohort of general oncology outpatients. Methods: Our prospective cohort consisted of patients aged ≥18 years who had a medical or gynecologic oncology encounter between March 1 and April 30, 2019 in either a tertiary academic practice or one of twelve community practices within a large academic cancer system. We used a retrospectively validated gradient-boosting ML algorithm, based on 559 structured electronic health record (EHR) variables, to predict 180-day mortality prior to each oncology encounter. For patients with multiple encounters, we selected the last encounter to assess performance. We assessed several performance metrics, including area under the receiver operating curve (AUC), area under the precision-recall curve (AUPRC), scaled Brier score (sBrier; a measure of calibration ranging from 0 [random] to 1 [perfect]), and positive predictive value (PPV). Results: Of 25,537 unique patients, median age was 64.4 (interquartile range 53.3 – 73.0), 76.8% were White, 56.5% were treated at a community center, and 4.1% died within 180 days. The ML algorithm had an AUC of 0.89 (95% confidence interval [CI] 0.88-0.90), AUPRC 0.34, and sBrier 0.29. At a prespecified threshold of 40%, observed 180-day mortality was 44.5% (95% CI 40.7 – 48.4%) in the high-risk group vs. 3.0% (95% CI 2.8% – 3.3%) in the low-risk group. There was an 85-fold difference in mortality (13.6% vs. 0.16%) in the top vs. bottom risk quartiles. The model was well-calibrated for mortality risks ≤40% and slightly under-calibrated for mortality risks > 40%. Performance varied across cancer types in the tertiary hospital but did not vary by race or practice type (Table). Conclusions: In this prospective cohort study among outpatients with cancer, a ML prognostic algorithm based on EHR data had better discrimination and calibration that published cancer-specific models. This is one of the first ML prognostic models to be prospectively validated in oncology. [Table: see text]
- 1 November 2019
- Published by American Society of Clinical Oncology (ASCO) in Journal of Clinical Oncology
- Vol. 37 (31_suppl) , 131
Derivation and implementation of a machine learning approach to prompt serious illness conversations among outpatients with cancer.
131 Background: Machine learning (ML) algorithms can accurately identify patients with cancer at risk of short-term mortality and facilitate timely conversations about treatment and end-of-life preferences. We developed, validated, and implemented a ML algorithm to predict mortality in a general oncology setting, using electronic health record (EHR) data prior to a clinic visit. Methods: Our cohort consisted of patients aged ≥18 years who had an encounter in outpatient oncology practices within a large academic health system between February 1st and July 1st, 2016. We randomly split the sample into training (70%) and validation (30%) cohorts at the patient-encounter level. We trained three ML algorithms to predict 180-day mortality and describe performance in the holdout validation cohort. From October 2018 to February 2019, we used the best-performing algorithm to generate weekly lists of high-risk patients at a single community oncology practice and studied the impact on rates of documented serious illness conversations (SICs). Results: Among 62,377 encounters used to train the algorithms, 7.4% involved a patient who died within 180 days. Gradient boosting and/or random forest outperformed logistic regression in all metrics (Table), and the gradient boosting model had superior discrimination and calibration. In the gradient boosting model, observed 180-day mortality was 45.5% (95% CI 39.0-52.3%) in the high-risk group vs. 3.3% (95% CI 2.9-3.7%) in the low-risk group. In a survey of oncology clinicians, 59% of patients flagged as high-risk were appropriate for a serious illness conversation in the upcoming week (response rate 52%). Five months after implementing the intervention, average monthly documented SICs increased by 23% (31.7 to 39). Conclusions: A ML algorithm based on EHR data accurately identified patients with cancer at risk of short-term mortality, was concordant with oncologists’ assessments, and was associated with more SICs. [Table: see text]
- 1 November 2019
- Published by American Society of Clinical Oncology (ASCO) in Journal of Clinical Oncology
- Vol. 37 (31_suppl) , 131
Derivation and implementation of a machine learning approach to prompt serious illness conversations among outpatients with cancer.
131 Background: Machine learning (ML) algorithms can accurately identify patients with cancer at risk of short-term mortality and facilitate timely conversations about treatment and end-of-life preferences. We developed, validated, and implemented a ML algorithm to predict mortality in a general oncology setting, using electronic health record (EHR) data prior to a clinic visit. Methods: Our cohort consisted of patients aged ≥18 years who had an encounter in outpatient oncology practices within a large academic health system between February 1st and July 1st, 2016. We randomly split the sample into training (70%) and validation (30%) cohorts at the patient-encounter level. We trained three ML algorithms to predict 180-day mortality and describe performance in the holdout validation cohort. From October 2018 to February 2019, we used the best-performing algorithm to generate weekly lists of high-risk patients at a single community oncology practice and studied the impact on rates of documented serious illness conversations (SICs). Results: Among 62,377 encounters used to train the algorithms, 7.4% involved a patient who died within 180 days. Gradient boosting and/or random forest outperformed logistic regression in all metrics (Table), and the gradient boosting model had superior discrimination and calibration. In the gradient boosting model, observed 180-day mortality was 45.5% (95% CI 39.0-52.3%) in the high-risk group vs. 3.3% (95% CI 2.9-3.7%) in the low-risk group. In a survey of oncology clinicians, 59% of patients flagged as high-risk were appropriate for a serious illness conversation in the upcoming week (response rate 52%). Five months after implementing the intervention, average monthly documented SICs increased by 23% (31.7 to 39). Conclusions: A ML algorithm based on EHR data accurately identified patients with cancer at risk of short-term mortality, was concordant with oncologists’ assessments, and was associated with more SICs. [Table: see text]
- Open Access
- 2 October 2019
- Published by American Medical Association (AMA) in JAMA Network Open
- Vol. 2 (10) , e1914051
Trends and Focus of Machine Learning Applications for Health Research
Machine learning for health care is a rapidly growing interdisciplinary field gaining interest in academia and practice.1-6 The purpose of this work was to perform quantitative and qualitative evaluations of the state of machine learning for health research. As the field evolves, analyses can elucidate research trends, behaviors, and future opportunities. In this work, we performed an analysis using the submitted and accepted manuscripts from the recent Third Annual Machine Learning for Health (ML4H) workshop at the 32nd Conference on Neural Information Processing Systems (NeurIPS) on December 8, 2018.7
- 2 October 2019
- Published by American Medical Association (AMA) in JAMA Network Open
- Vol. 2 (10) , e1914051
Open AccessTrends and Focus of Machine Learning Applications for Health Research
Machine learning for health care is a rapidly growing interdisciplinary field gaining interest in academia and practice.1-6 The purpose of this work was to perform quantitative and qualitative evaluations of the state of machine learning for health research. As the field evolves, analyses can elucidate research trends, behaviors, and future opportunities. In this work, we performed an analysis using the submitted and accepted manuscripts from the recent Third Annual Machine Learning for Health (ML4H) workshop at the 32nd Conference on Neural Information Processing Systems (NeurIPS) on December 8, 2018.7