Boosting the accuracy of existing models by updating and extending: using a multicenter COVID-19 ICU cohort as a proxy
Authors:
- Daniek A. M. MeijsDMDaniek A. M. Meijs
- Laure WynantsLWLaure Wynants
- Sander M. J. van KuijkSKSander M. J. van Kuijk
- Jannet Mehagnoul-SchipperJMJannet Mehagnoul-Schipper
- Björn StesselBSBjörn Stessel
- Eline G. M. CoxECEline G. M. Cox
- Jan-Willem E. M. SelsJSJan-Willem E. M. Sels
- Luc J. M. SmitsLSLuc J. M. Smits
- Johannes BickenbachJBJohannes Bickenbach
Open Access
- 1 November 2024
- journal article
- research article
- Published by Springer Nature in Scientific Reports
- Vol. 14 (1) , 1-11
Abstract
Most published prediction models for Coronavirus Disease 2019 (COVID-19) were poorly reported, at high risk of bias, and heterogeneous in model performance. To tackle methodological challenges faced in previous prediction studies, we investigated whether model updating and extending improves mortality prediction, using the Intensive Care Unit (ICU) as a proxy. All COVID-19 patients admitted to seven ICUs in the Euregio-Meuse Rhine during the first pandemic wave were included. The 4C Mortality and SEIMC scores were selected as promising prognostic models from an external validation study. Five predictors could be estimated based on cohort size. TRIPOD guidelines were followed and logistic regression analyses with the linear predictor, APACHE II score, and country were performed. Bootstrapping with backward selection was applied to select variables for the final model. Additionally, shrinkage was performed. Model discrimination was displayed as optimism-corrected areas under the ROC curve and calibration by calibration slopes and plots. The mortality rate of the 551 included patients was 36%. Discrimination of the 4C Mortality and SEIMC scores increased from 0.70 to 0.74 and 0.70 to 0.73 and calibration plots improved compared to the original models after updating and extending. Mortality prediction can be improved after updating and extending of promising models.Keywords
This publication has 33 references indexed in Scilit:
- External Validation and Recalibration of the CURB-65 and PSI for Predicting 30-Day Mortality and Critical Care Intervention in Multiethnic Patients with COVID-19International Journal of Infectious Diseases, 2021
- External validation of clinical prediction models: simulation-based sample size calculations were more reliable than rules-of-thumbJournal of Clinical Epidemiology, 2021
- Clinical Prediction Models2019
- A calibration hierarchy for risk models was defined: from utopia to empirical dataJournal of Clinical Epidemiology, 2016
- Sample size considerations for the external validation of a multivariable prognostic model: a resampling studyStatistics in Medicine, 2015
- Regression Modeling Strategies2015
- Multiple imputation using chained equations: Issues and guidance for practiceStatistics in Medicine, 2010
- Clinical Prediction Models2009
- Substantial effective sample sizes were required for external validation studies of predictive logistic regression modelsJournal of Clinical Epidemiology, 2005
- A new method of classifying prognostic comorbidity in longitudinal studies: Development and validationJournal of Chronic Diseases, 1987