A Machine Learning Approach for Mortality Prediction in COVID-19 Pneumonia: Development and Evaluation of the Piacenza Score
Authors:
- Geza HalaszGHGeza Halasz
- Michela SpertiMSMichela Sperti
- Matteo VillaniMVMatteo Villani
- Umberto MichelucciUmberto Michelucci
- Piergiuseppe AgostoniPAPiergiuseppe Agostoni
- Andrea BiagiABAndrea Biagi
- Luca RossiLRLuca Rossi
- Andrea BottiABAndrea Botti
- Chiara MariCMChiara Mari
- Marco MaccariniMMMarco Maccarini
- Filippo PuraFPFilippo Pura
- Loris RovedaLRLoris Roveda
Open Access
- 31 May 2021
- journal article
- research article
- Published by JMIR Publications Inc. in Journal of Medical Internet Research
- Vol. 23 (5) , e29058
Abstract
Background: Several models have been developed to predict mortality in patients with COVID-19 pneumonia, but only a few have demonstrated enough discriminatory capacity. Machine learning algorithms represent a novel approach for the data-driven prediction of clinical outcomes with advantages over statistical modeling. Objective: We aimed to develop a machine learning–based score—the Piacenza score—for 30-day mortality prediction in patients with COVID-19 pneumonia. Methods: The study comprised 852 patients with COVID-19 pneumonia, admitted to the Guglielmo da Saliceto Hospital in Italy from February to November 2020. Patients’ medical history, demographics, and clinical data were collected using an electronic health record. The overall patient data set was randomly split into derivation and test cohorts. The score was obtained through the naïve Bayes classifier and externally validated on 86 patients admitted to Centro Cardiologico Monzino (Italy) in February 2020. Using a forward-search algorithm, 6 features were identified: age, mean corpuscular hemoglobin concentration, PaO2/FiO2 ratio, temperature, previous stroke, and gender. The Brier index was used to evaluate the ability of the machine learning model to stratify and predict the observed outcomes. A user-friendly website was designed and developed to enable fast and easy use of the tool by physicians. Regarding the customization properties of the Piacenza score, we added a tailored version of the algorithm to the website, which enables an optimized computation of the mortality risk score for a patient when some of the variables used by the Piacenza score are not available. In this case, the naïve Bayes classifier is retrained over the same derivation cohort but using a different set of patient characteristics. We also compared the Piacenza score with the 4C score and with a naïve Bayes algorithm with 14 features chosen a priori. Results: The Piacenza score exhibited an area under the receiver operating characteristic curve (AUC) of 0.78 (95% CI 0.74-0.84, Brier score=0.19) in the internal validation cohort and 0.79 (95% CI 0.68-0.89, Brier score=0.16) in the external validation cohort, showing a comparable accuracy with respect to the 4C score and to the naïve Bayes model with a priori chosen features; this achieved an AUC of 0.78 (95% CI 0.73-0.83, Brier score=0.26) and 0.80 (95% CI 0.75-0.86, Brier score=0.17), respectively. Conclusions: Our findings demonstrated that a customizable machine learning–based score with a purely data-driven selection of features is feasible and effective for the prediction of mortality among patients with COVID-19 pneumonia.Keywords
All Related Versions
This publication has 25 references indexed in Scilit:
- Artificial Intelligence (AI) applications for COVID-19 pandemicDiabetes & Metabolic Syndrome: Clinical Research & Reviews, 2020
- Risk stratification in pulmonary arterial hypertension using Bayesian analysisEuropean Respiratory Journal, 2020
- Clinical Characteristics of Coronavirus Disease 2019 in ChinaNew England Journal of Medicine, 2020
- Prediction models for diagnosis and prognosis of covid-19: systematic review and critical appraisalBMJ, 2020
- Clinical course and risk factors for mortality of adult inpatients with COVID-19 in Wuhan, China: a retrospective cohort studyThe Lancet, 2020
- Meta‐Analysis Global Group in Chronic (MAGGIC) Heart Failure Risk Score: Validation of a Simple Tool for the Prediction of Morbidity and Mortality in Heart Failure With Preserved Ejection FractionJournal of the American Heart Association, 2018
- Prediction of In‐hospital Mortality in Emergency Department Patients With Sepsis: A Local Big Data–Driven, Machine Learning ApproachAcademic Emergency Medicine, 2015
- A Naive Bayes machine learning approach to risk prediction using censored, time‐to‐event dataStatistics in Medicine, 2015
- The Seattle Heart Failure ModelCirculation, 2006
- Floating search methods in feature selectionPattern Recognition Letters, 1994