A new confidence interval based on the theory of U-statistics for the area under the curve
Open Access
- 9 October 2025
- journal article
- Published by VTeX in Modern Stochastics: Theory and Applications
- p. 57-82
Abstract
The area under the receiver operating characteristic curve (AUC) is a suitable measure for the quality of classification algorithms. Here we use the theory of U-statistics in order to derive new confidence intervals for it. The new confidence intervals take into account that only the total sample size used to calculate the AUC can be controlled, while the number of members of the case group and the number of members of the control group are random. We show that the new confidence intervals can not only be used in order to evaluate the quality of the fitted model, but also to judge the quality of the classification algorithm itself. We would like to take this opportunity to show that two popular confidence intervals for the AUC, namely DeLong’s interval and the Mann–Whitney intervals due to Sen, coincide.Keywords
This publication has 10 references indexed in Scilit:
- Confidence intervals of prediction accuracy measures for multivariable prediction models based on the bootstrap‐based optimism correction methodsStatistics in Medicine, 2021
- On the limiting distribution of sample central momentsAnnals of the Institute of Statistical Mathematics, 2018
- Computationally efficient confidence intervals for cross-validated area under the ROC curve estimatesElectronic Journal of Statistics, 2015
- A modified Wald interval for the area under the ROC curve (AUC) in diagnostic case-control studiesBMC Medical Research Methodology, 2014
- Comparison of non-parametric confidence intervals for the area under the ROC curve of a continuous-scale diagnostic testStatistical Methods in Medical Research, 2007
- Modern Applied U‐Statistics2007
- The Statistical Evaluation of Medical Tests for Classification and Prediction2003
- Bootstrap confidence intervals: when, which, what? A practical guide for medical statisticiansStatistics in Medicine, 2000
- Convergence of Probability Measures1999
- Comparing the Areas under Two or More Correlated Receiver Operating Characteristic Curves: A Nonparametric ApproachBiometrics, 1988