A study on cost behaviors of binary classification measures in class-imbalanced problems
Preprint
Abstract
This work investigates into cost behaviors of binary classification measures in a background of class-imbalanced problems. Twelve performance measures are studied, such as F measure, G-means in terms of accuracy rates, and of recall and precision, balance error rate (BER), Matthews correlation coefficient (MCC), Kappa coefficient, etc. A new perspective is presented for those measures by revealing their cost functions with respect to the class imbalance ratio. Basically, they are described by four types of cost functions. The functions provides a theoretical understanding why some measures are suitable for dealing with class-imbalanced problems. Based on their cost functions, we are able to conclude that G-means of accuracy rates and BER are suitable measures because they show "proper" cost behaviors in terms of "a misclassification from a small class will cause a greater cost than that from a large class". On the contrary, F1 measure, G-means of recall and precision, MCC and Kappa coefficient measures do not produce such behaviors so that they are unsuitable to serve our goal in dealing with the problems properly.Keywords
All Related Versions
This publication has 14 references indexed in Scilit:
- A New Strategy of Cost-Free Learning in the Class Imbalance ProblemIEEE Transactions on Knowledge and Data Engineering, 2014
- Adjusted F-measure and kernel scaling for imbalanced data learningInformation Sciences, 2014
- Measures for ranking cell trackers without manual validationPattern Recognition, 2013
- Novel classifier scheme for imbalanced problemsPattern Recognition Letters, 2013
- A Quadratic Mean based Supervised Learning Model for Managing Data Skewness2011
- An experimental comparison of performance measures for classificationPattern Recognition Letters, 2009
- A balanced accuracy function for epistasis modeling in imbalanced datasets using multifactor dimensionality reductionGenetic Epidemiology, 2007
- EVALUATION OF CLASSIFIERS FOR AN UNEVEN CLASS DISTRIBUTION PROBLEMApplied Artificial Intelligence, 2006
- Using AUC and accuracy in evaluating learning algorithmsIEEE Transactions on Knowledge and Data Engineering, 2005
- Assessing the accuracy of prediction algorithms for classification: an overviewBioinformatics, 2000