David van Klaveren

Concordance measures are frequently used for assessing the discriminative ability of risk prediction models. The interpretation of estimated concordance at external validation is difficult if the case‐mix differs from the model development setting. We aimed to develop a concordance measure that provides insight into the influence of case‐mix heterogeneity and is robust to censoring of time‐to‐event data.We first derived a model‐based concordance (mbc) measure that allows for quantification of the influence of case‐mix heterogeneity on discriminative ability of proportional hazards and logistic regression models. This mbc can also be calculated including a regression slope that calibrates the predictions at external validation (c‐mbc), hence assessing the influence of overall regression coefficient validity on discriminative ability. We derived variance formulas for both mbc and c‐mbc. We compared the mbc and the c‐mbc with commonly used concordance measures in a simulation study and in two external validation settings.The mbc was asymptotically equivalent to a previously proposed resampling‐based case‐mix corrected c‐index. The c‐mbc remained stable at the true value with increasing proportions of censoring, while Harrell's c‐index and to a lesser extent Uno's concordance measure increased unfavorably. Variance estimates of mbc and c‐mbc were well in agreement with the simulated empirical variances.We conclude that the mbc is an attractive closed‐form measure that allows for a straightforward quantification of the expected change in a model's discriminative ability due to case‐mix heterogeneity. The c‐mbc also reflects regression coefficient validity and is a censoring‐robust alternative for the c‐index when the proportional hazards assumption holds. Copyright © 2016 John Wiley & Sons, Ltd.

Measurement of health-related quality of life (HRQL) is essential to quantify the subjective burden of traumatic brain injury (TBI) in survivors. We performed a systematic review of HRQL studies in TBI to evaluate study design, instruments used, methodological quality, and outcome. Fifty-eight studies were included, showing large variation in HRQL instruments and assessment time points used. The Short Form-36 (SF-36) was most frequently used. A high prevalence of health problems during and after the first year of TBI was a common finding of the studies included. In the long term, patients with a TBI still showed large deficits from full recovery compared to population norms. Positive results for internal consistency and interpretability of the SF-36 were reported in validity studies. The Quality of Life after Brain Injury instrument (QOLIBRI), European Brain Injury Questionnaire (EBIQ), Child Health Questionnaire (CHQ), and the World Health Organization Quality of Life short version (WHOQOL-BREF) showed positive results, but evidence was limited. Meta-analysis of SF-36 showed that TBI outcome is heterogeneous, encompassing a broad spectrum of HRQL, with most problems reported in the physical, emotional, and social functioning domain. The use of SF-36 in combination with a TBI-specific instrument, i.e., QOLIBRI, seems promising. Consensus on preferred methodologies of HRQL measurement in TBI would facilitate comparability across studies, resulting in improved insights in recovery patterns and better estimates of the burden of TBI.

The discriminative ability of a risk model is often measured by Harrell’s concordance-index (c-index). The c-index estimates for two randomly chosen subjects the probability that the model predicts a higher risk for the subject with poorer outcome (concordance probability). When data are clustered, as in multicenter data, two types of concordance are distinguished: concordance in subjects from the same cluster (within-cluster concordance probability) and concordance in subjects from different clusters (between-cluster concordance probability). We argue that the within-cluster concordance probability is most relevant when a risk model supports decisions within clusters (e.g. who should be treated in a particular center). We aimed to explore different approaches to estimate the within-cluster concordance probability in clustered data. We used data of the CRASH trial (2,081 patients clustered in 35 centers) to develop a risk model for mortality after traumatic brain injury. To assess the discriminative ability of the risk model within centers we first calculated cluster-specific c-indexes. We then pooled the cluster-specific c-indexes into a summary estimate with different meta-analytical techniques. We considered fixed effect meta-analysis with different weights (equal; inverse variance; number of subjects, events or pairs) and random effects meta-analysis. We reflected on pooling the estimates on the log-odds scale rather than the probability scale. The cluster-specific c-index varied substantially across centers (IQR = 0.70-0.81; I ² = 0.76 with 95% confidence interval 0.66 to 0.82). Summary estimates resulting from fixed effect meta-analysis ranged from 0.75 (equal weights) to 0.84 (inverse variance weights). With random effects meta-analysis – accounting for the observed heterogeneity in c-indexes across clusters – we estimated a mean of 0.77, a between-cluster variance of 0.0072 and a 95% prediction interval of 0.60 to 0.95. The normality assumptions for derivation of a prediction interval were better met on the probability than on the log-odds scale. When assessing the discriminative ability of risk models used to support decisions at cluster level we recommend meta-analysis of cluster-specific c-indexes. Particularly, random effects meta-analysis should be considered.