A closed max‐t test for multiple comparisons of areas under the ROC curve
- 18 November 2020
- journal article
- research article
- Published by Oxford University Press (OUP) in Biometrics
- Vol. 78 (1) , 352-363
Abstract
Comparing areas under the ROC curve (AUCs) is a popular approach to compare prognostic biomarkers. The aim of this paper is to present an efficient method to control the family‐wise error rate when multiple comparisons are performed. We suggest to combine the max‐t test and the closed testing procedures. We build on previous work on asymptotic results for ROC curves and on general multiple testing methods to efficiently take into account both the correlations between the test statistics and the logical constraints between the null hypotheses. The proposed method results in an uniformly more powerful procedure than both the single‐step max‐t test procedure and popular stepwise extensions of the Bonferroni procedure, such as Bonferroni–Holm. As demonstrated in this paper, the method can be applied in most usual contexts, including the time‐dependent context with right censored data. We show how the method works in practice through a motivating example where we compare several psychometric scores to predict the t‐year risk of Alzheimer's disease. The example illustrates several multiple testing settings and demonstrates the advantage of using the proposed methods over common alternatives. R code has been made available to facilitate the use of the methods by others.Keywords
Funding Information
- Fondation Bettencourt Schueller
This publication has 20 references indexed in Scilit:
- A Versatile Method for Confirmatory Evaluation of the Effects of a Covariate in Multiple ModelsJournal of the Royal Statistical Society Series C: Applied Statistics, 2011
- Report of the task force on designing clinical trials in early (predementia) ADNeurology, 2011
- Estimation methods for time‐dependent AUC models with survival dataCanadian Journal of Statistics, 2009
- Prodromal Alzheimer's disease: Successive emergence of the clinical symptomsAnnals of Neurology, 2008
- Simultaneous Inference in General Parametric ModelsBiometrical Journal, 2008
- Multiple Testing of General ContrastsJournal of the American Statistical Association, 2007
- Sensitivity of Four Psychometric Tests to Measure Cognitive Changes in Brain Aging-Population-based StudiesAmerican Journal of Epidemiology, 2006
- Multiple comparisons with the best ROC curve2004
- Multiple Testing of General Contrasts Using Logical Constraints and CorrelationsJournal of the American Statistical Association, 1997
- Modified Sequentially Rejective Multiple Test ProceduresJournal of the American Statistical Association, 1986