Derivation and assessment of risk prediction models using case-cohort data
Open Access
- 13 September 2013
- journal article
- research article
- Published by Springer Nature in BMC Medical Research Methodology
- Vol. 13 (1) , 113
Abstract
Background: Case-cohort studies are increasingly used to quantify the association of novel factors with disease risk. Conventional measures of predictive ability need modification for this design. We show how Harrell’s C-index, Royston’s D, and the category-based and continuous versions of the net reclassification index (NRI) can be adapted.Methods: We simulated full cohort and case-cohort data, with sampling fractions ranging from 1% to 90%, using covariates from a cohort study of coronary heart disease, and two incidence rates. We then compared the accuracy and precision of the proposed risk prediction metrics.Results: The C-index and D must be weighted in order to obtain unbiased results. The NRI does not need modification, provided that the relevant non-subcohort cases are excluded from the calculation. The empirical standard errors across simulations were consistent with analytical standard errors for the C-index and D but not for the NRI. Good relative efficiency of the prediction metrics was observed in our examples, provided the sampling fraction was above 40% for the C-index, 60% for D, or 30% for the NRI. Stata code is made available.Conclusions: Case-cohort designs can be used to provide unbiased estimates of the C-index, D measure and NRI.Keywords
This publication has 28 references indexed in Scilit:
- C-Reactive Protein, Fibrinogen, and Cardiovascular Disease PredictionNew England Journal of Medicine, 2012
- Literature-Based Genetic Risk Scores for Coronary Heart DiseaseCirculation: Cardiovascular Genetics, 2012
- Risk Prediction Measures for Case-Cohort and Nested Case-Control Designs: An Application to Cardiovascular DiseaseAmerican Journal of Epidemiology, 2012
- Extensions of net reclassification improvement calculations to measure usefulness of new biomarkersStatistics in Medicine, 2010
- Assessing the Performance of Prediction ModelsEpidemiology, 2010
- Prognostic Models With Competing RisksEpidemiology, 2009
- EPIC-Heart: The cardiovascular component of a prospective study of nutritional, lifestyle and biological factors in 520,000 middle-aged participants from 10 European countriesEuropean Journal of Epidemiology, 2007
- An Assessment of Incremental Coronary Risk Prediction Using C-Reactive Protein and Other Novel Risk MarkersArchives of internal medicine (1960), 2006
- Estimation of time‐dependent area under the ROC curve for long‐term risk predictionStatistics in Medicine, 2005
- A new measure of prognostic separation in survival dataStatistics in Medicine, 2004