Comparison of likelihood penalization and variance decomposition approaches for clinical prediction models: A simulation study
Open Access
- 18 May 2023
- journal article
- research article
- Published by Wiley in Biometrical Journal
- Vol. 66 (1) , e2200108
Abstract
Logistic regression is one of the most commonly used approaches to develop clinical risk prediction models. Developers of such models often rely on approaches that aim to minimize the risk of overfitting and improve predictive performance of the logistic model, such as through likelihood penalization and variance decomposition techniques. We present an extensive simulation study that compares the out‐of‐sample predictive performance of risk prediction models derived using the elastic net, with Lasso and ridge as special cases, and variance decomposition techniques, namely, incomplete principal component regression and incomplete partial least squares regression. We varied the expected events per variable, event fraction, number of candidate predictors, presence of noise predictors, and the presence of sparse predictors in a full‐factorial design. Predictive performance was compared on measures of discrimination, calibration, and prediction error. Simulation metamodels were derived to explain the performance differences within model derivation approaches. Our results indicate that, on average, prediction models developed using penalization and variance decomposition approaches outperform models developed using ordinary maximum likelihood estimation, with penalization approaches being consistently superior over the variance decomposition approaches. Differences in performance were most pronounced on the calibration of the model. Performance differences regarding prediction error and concordance statistic outcomes were often small between approaches. The use of likelihood penalization and variance decomposition techniques methods was illustrated in the context of peripheral arterial disease.Keywords
Funding Information
- ZonMw (917.16.430)
This publication has 28 references indexed in Scilit:
- Reporting and Methods in Clinical Prediction Research: A Systematic ReviewPLOS Medicine, 2012
- Assessing the Performance of Prediction ModelsEpidemiology, 2010
- Fisher Lecture: Dimension Reduction in RegressionStatistical Science, 2007
- Validation and updating of predictive logistic regression models: a study on sample size and shrinkageStatistics in Medicine, 2004
- Prognostic Modeling with Logistic Regression AnalysisMedical Decision Making, 2001
- Validation of Probabilistic PredictionsMedical Decision Making, 1993
- Summarizing Monte Carlo Results in Methodological Research: The One- and Two-Factor Fixed Effects ANOVA CasesJournal of Educational Statistics, 1992
- Ridge Estimators in Logistic RegressionJournal of the Royal Statistical Society Series C: Applied Statistics, 1992
- Regression modelling strategies for improved prognostic predictionStatistics in Medicine, 1984
- Ridge Regression: Biased Estimation for Nonorthogonal ProblemsTechnometrics, 1970