Binary classification performance measures/metrics: A comprehensive visualized roadmap to gain new insights
Authors:
- 1 October 2017
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 821-826
Abstract
Binary classification is one of the most frequent studies in applied machine learning problems in various domains, from medicine to biology to meteorology to malware analysis. Many researchers use some performance metrics in their classification studies to report their success. However, the literature has shown a widespread confusion about the terminology and ignorance of the fundamental aspects behind metrics. This paper clarifies the confusing terminology, suggests formal rules to distinguish between measures and metrics for the first time, and proposes a new comprehensive visualized roadmap in a leveled structure for 22 measures and 22 metrics for exploring binary classification performance. Additionally, we introduced novel concepts such as canonical notation, duality, and complementation for measures/metrics, and suggested two new canonical base measures simplifying equations. It is expected that the study will guide other studies to have standardized approach to performance metrics for machine learning based solutions.Keywords
This publication has 13 references indexed in Scilit:
- On the order equivalence relation of binary association measuresInternational Journal of Applied Mathematics and Computer Science, 2015
- 100% Classification Accuracy Considered Harmful: The Normalized Information Transfer Factor Explains the Accuracy ParadoxPLOS ONE, 2014
- How to evaluate an agent's behavior to infrequent events?—Reliable performance estimation insensitive to class distributionFrontiers in Computational Neuroscience, 2014
- Analysis of Bayesian classification‐based approaches for Android malware detectionIET Information Security, 2014
- A systematic analysis of performance measures for classification tasksInformation Processing & Management, 2009
- Diagnostic methods I: sensitivity, specificity, and other measures of accuracyKidney International, 2009
- The Business Impact of Predictive Analytics2007
- Classification-algorithm evaluation: Five performance measures based onconfusion matricesJournal of Clinical Monitoring and Computing, 1995
- COEFFICIENTS OF ASSOCIATION AND SIMILARITY, BASED ON BINARY (PRESENCE‐ABSENCE) DATA: AN EVALUATIONBiological Reviews, 1982
- MEASURES OF ASSOCIATION AMONG VARIABLES1973