Improved Prediction by Dynamic Modeling

Authors:
  • DNDaan Nieboer
    Daan Nieboer
  • YVYvonne Vergouwe
    Yvonne Vergouwe
  • MVMichel I.M. Versteegh
    Michel I.M. Versteegh
  • LNLuc Noyez
    Luc Noyez
  • AVAlexander B.A. Vonk
    Alexander B.A. Vonk
  • ESEwout W. Steyerberg
    Ewout W. Steyerberg
  • JTJohanna J.M. Takkenberg
    Johanna J.M. Takkenberg
Publisher website
Google Scholar
Abstract
Background—: The predictive performance of static risk prediction models such as EuroSCORE deteriorates over time. We aimed to explore different methods for continuous updating of EuroSCORE (dynamic modeling) to improve risk prediction. Methods and Results—: Data on adult cardiac surgery from 2007 to 2012 (n=95 240) were extracted from the Netherlands Association for Cardio-Thoracic Surgery database. The logistic EuroSCORE predicting in-hospital death was updated using 6 methods: recalibrating the intercept of the logistic regression model; recalibrating the intercept and joint effects of the prognostic factors; re-estimating all prognostic factor effects, re-estimating all prognostic factor effects, and applying shrinkage of the estimates; applying a test procedure to select either of these; and a Bayesian learning strategy. Models were updated with 1 or 3 years of data, in all cardiac surgery or within operation subgroups. Performance was tested in the subsequent year according to discrimination (area under the receiver operating curve, area under the curve) and calibration (calibration slope and calibration-in-the-large). Compared with the original EuroSCORE, all updating methods resulted in improved calibration-in-the-large (range −0.17 to 0.04 versus −1.13 to −0.97, ideally 0.0). Calibration slope (range 0.92–1.15) and discrimination (area under the curve range 0.83–0.87) were similar across methods. In small subgroups, such as aortic valve replacement and aortic valve replacement+coronary artery bypass grafting, extensive updating using 1 year of data led to poorer performance than using the original EuroSCORE. The choice of updating method had little effect on benchmarking results of all cardiac surgery. Conclusions—: Several methods for dynamic modeling may result in good discrimination and superior calibration compared with the original EuroSCORE. For large populations, all methods are appropriate. For smaller subgroups, it is recommended to use data from multiple years or a Bayesian approach.

This publication has 28 references indexed in Scilit: