A framework for meta-analysis of prediction model studies with binary and time-to-event outcomes
Top Cited Papers
Open Access
- 23 July 2018
- journal article
- research article
- Published by SAGE Publications in Statistical Methods in Medical Research
- Vol. 28 (9) , 2768-2786
Abstract
It is widely recommended that any developed—diagnostic or prognostic—prediction model is externally validated in terms of its predictive performance measured by calibration and discrimination. When multiple validations have been performed, a systematic review followed by a formal meta-analysis helps to summarize overall performance across multiple settings, and reveals under which circumstances the model performs suboptimal (alternative poorer) and may need adjustment. We discuss how to undertake meta-analysis of the performance of prediction models with either a binary or a time-to-event outcome. We address how to deal with incomplete availability of study-specific results (performance estimates and their precision), and how to produce summary estimates of the c-statistic, the observed:expected ratio and the calibration slope. Furthermore, we discuss the implementation of frequentist and Bayesian meta-analysis methods, and propose novel empirically-based prior distributions to improve estimation of between-study heterogeneity in small samples. Finally, we illustrate all methods using two examples: meta-analysis of the predictive performance of EuroSCORE II and of the Framingham Risk Score. All examples and meta-analysis models have been implemented in our newly developed R package “metamisc”.Keywords
Funding Information
- Netherlands Organization for Scientific Research (91617050)
- Cochrane Methods Innovation Funds Round 2 (MTH001F)
This publication has 63 references indexed in Scilit:
- Prognosis Research Strategy (PROGRESS) 3: Prognostic Model ResearchPLOS Medicine, 2013
- Interpreting the concordance statistic of a logistic regression model: relation to the variance and odds ratio of a continuous explanatory variableBMC Medical Research Methodology, 2012
- Reporting and Methods in Clinical Prediction Research: A Systematic ReviewPLOS Medicine, 2012
- Original and REGICOR Framingham Functions in a Nondiabetic Population of a Spanish Health Care Center: A Validation StudyThe Annals of Family Medicine, 2011
- External Validity of Risk Models: Use of Benchmark Values to Disentangle a Case-Mix Effect From Incorrect CoefficientsAmerican Journal of Epidemiology, 2010
- Assessing the Performance of Prediction ModelsEpidemiology, 2010
- A Re-Evaluation of Random-Effects Meta-AnalysisJournal of the Royal Statistical Society Series A: Statistics in Society, 2008
- A new measure of prognostic separation in survival dataStatistics in Medicine, 2004
- Advanced methods in meta‐analysis: multivariate approach and meta‐regressionStatistics in Medicine, 2002
- Meta-analysis in clinical trialsControlled Clinical Trials, 1986