ROC analysis with multiple classes and multiple tests: methodology and its application in microarray studies
Open Access
- 27 February 2008
- journal article
- research article
- Published by Oxford University Press (OUP) in Biostatistics
- Vol. 9 (3) , 566-576
Abstract
The accuracy of a single diagnostic test for binary outcome can be summarized by the area under the receiver operating characteristic (ROC) curve. Volume under the surface and hypervolume under the manifold have been proposed as extensions for multiple class diagnosis (Scurfield, 1996, 1998). However, the lack of simple inferential procedures for such measures has limited their practical utility. Part of the difficulty is that calculating such quantities may not be straightforward, even with a single test. The decision rule used to generate the ROC surface requires class probability assessments, which are not provided by the tests. We develop a method based on estimating the probabilities via some procedure, for example, multinomial logistic regression. Bootstrap inferences are proposed to account for variability in estimating the probabilities and perform well in simulations. The ROC measures are compared to the correct classification rate, which depends heavily on class prevalences. An example of tumor classification with microarray data demonstrates that this property may lead to substantially different analyses. The ROC-based analysis yields notable decreases in model complexity over previous analyses.Keywords
This publication has 13 references indexed in Scilit:
- Prevalence‐dependent diagnostic accuracy measuresStatistics in Medicine, 2007
- Multicategory Support Vector MachinesJournal of the American Statistical Association, 2004
- Combining Several Screening Tests: Optimality of the Risk ScoreBiometrics, 2002
- Comparison of Discrimination Methods for the Classification of Tumors Using Gene Expression DataJournal of the American Statistical Association, 2002
- Classification and diagnostic prediction of cancers using gene expression profiling and artificial neural networksNature Medicine, 2001
- Comparing Three-class Diagnostic Tests by Three-way ROC AnalysisMedical Decision Making, 2000
- Three-way ROCsMedical Decision Making, 1999
- Ratio-based decisions and the quantitative analysis of cDNA microarray imagesJournal of Biomedical Optics, 1997
- A Class of Statistics with Asymptotically Normal DistributionThe Annals of Mathematical Statistics, 1948
- THE USE OF MULTIPLE MEASUREMENTS IN TAXONOMIC PROBLEMSAnnals of Eugenics, 1936