Comparing the Predictive Powers of Survival Models Using Harrell's C or Somers’ D
- 1 September 2010
- journal article
- research article
- Published by SAGE Publications in The Stata Journal: Promoting communications on statistics and Stata
- Vol. 10 (3) , 339-358
Abstract
Medical researchers frequently make statements that one model predicts survival better than another, and they are frequently challenged to provide rigorous statistical justification for those statements. Stata provides the estat concordance command to calculate the rank parameters Harrell's C and Somers’ D as measures of the ordinal predictive power of a model. However, no confidence limits or p-values are provided to compare the predictive power of distinct models. The somersd package, downloadable from Statistical Software Components, can provide such confidence intervals, but they should not be taken seriously if they are calculated in the dataset in which the model was fit. Methods are demonstrated for fitting alternative models to a training set of data, and then measuring and comparing their predictive powers by using out-of-sample prediction and somersd in a test set to produce statistically sensible confidence intervals and p-values for the differences between the predictive powers of different models.Keywords
This publication has 9 references indexed in Scilit:
- Confidence Intervals for Rank Statistics: Percentile Slopes, Differences, and RatiosThe Stata Journal: Promoting communications on statistics and Stata, 2006
- Confidence Intervals for Rank Statistics: Somers’ D and ExtensionsThe Stata Journal: Promoting communications on statistics and Stata, 2006
- Efficient Calculation of Jackknife Confidence Intervals for Rank StatisticsJournal of Statistical Software, 2006
- Parameters behind “Nonparametric” Statistics: Kendall's tau, Somers’ D and Median DifferencesThe Stata Journal: Promoting communications on statistics and Stata, 2002
- MULTIVARIABLE PROGNOSTIC MODELS: ISSUES IN DEVELOPING MODELS, EVALUATING ASSUMPTIONS AND ADEQUACY, AND MEASURING AND REDUCING ERRORSStatistics in Medicine, 1996
- A Confidence Interval for Pr(X < Y) - Pr(X > Y) Estimated from Simple Cluster Samples1995
- The central role of the propensity score in observational studies for causal effectsBiometrika, 1983
- THE SIGNIFICANCE OF RANK CORRELATIONS WHERE PARENTAL CORRELATION EXISTSBiometrika, 1947
- The Logic of Inductive InferenceJournal of the Royal Statistical Society, 1935