Optimal threshold estimation for binary classifiers using game theory
Open Access
- 15 December 2016
- journal article
- Published by F1000 Research Ltd in F1000Research
- Vol. 5, 2762
Abstract
Many bioinformatics algorithms can be understood as binary classifiers. They are usually trained by maximizing the area under the receiver operating characteristic (ROC) curve. On the other hand, choosing the best threshold for practical use is a complex task, due to uncertain and context-dependent skews in the abundance of positives in nature and in the yields/costs for correct/incorrect classification. We argue that considering a classifier as a player in a zero-sum game allows us to use the minimax principle from game theory to determine the optimal operating point. The proposed classifier threshold corresponds to the intersection between the ROC curve and the descending diagonal in ROC space and yields a minimax accuracy of 1-FPR. Our proposal can be readily implemented in practice, and reveals that the empirical condition for threshold estimation of “specificity equals sensitivity” maximizes robustness against uncertainties in the abundance of positives in nature and classification costs.Keywords
This publication has 8 references indexed in Scilit:
- Towards High-throughput Immunomics for Infectious Diseases: Use of Next-generation Peptide Microarrays for Rapid Discovery and Mapping of Antigenic DeterminantsMolecular & Cellular Proteomics, 2015
- An introduction to ROC analysisPattern Recognition Letters, 2006
- Optimal robust classifiersPattern Recognition, 2005
- Assessing computational tools for the discovery of transcription factor binding sitesNature Biotechnology, 2005
- Information theory approach to learning of the perceptron rulePhysical Review E, 2001
- Better Decisions through ScienceScientific American, 2000
- Statistical pattern recognition: a reviewIEEE Transactions on Pattern Analysis and Machine Intelligence, 2000
- The statistical mechanics of learning a ruleReviews of Modern Physics, 1993