Discrete Box-Constrained Minimax Classifier for Uncertain and Imbalanced Class Proportions
Preprint
Abstract
The goal of this paper is to build a supervised classifier addressing the following difficulties which commonly appear in safety-critical applications: imbalanced datasets, uncertain class proportions, dependencies between some features, presence of both numeric and categorical features, and arbitrary loss functions provided by experts. Many works have shown that discretizing the numeric features is relevant for dealing with mixed attributes. Thus, we develop a novel minimax classifier algorithm, designed for processing discrete or discretized features, which addresses all the previously mentioned issues. The usual minimax criterion derives from the computation of the class proportions which maximize the empirical Bayes risk over the probabilistic simplex. However, it can be potentially too pessimistic when the least favorable priors appear unrealistic or its risk of error becomes too high. In this case, under the assumption that the experts are able to provide independent bounds on some class proportions, our approach takes into account these constraints to decrease the risk of error. The resulting box-constrained minimax classifier appears as a trade-off between the discrete Bayes classifier and the usual minimax classifier.Keywords
All Related Versions
This publication has 19 references indexed in Scilit:
- A filtered bucket-clustering method for projection onto the simplex and the $\ell _1$ ballMathematical Programming, 2019
- Constructive minimax classification of discrete observations with arbitrary loss functionSignal Processing, 2017
- Tutorial on practical tips of the most influential data preprocessing algorithms in data miningKnowledge-Based Systems, 2016
- Discretization for naive-Bayes learning: managing discretization bias and varianceMachine Learning, 2008
- Ten Lectures on Statistical and Structural Pattern Recognition2002
- An overview of statistical learning theoryIEEE Transactions on Neural Networks, 1999
- On the projected subgradient method for nonsmooth convex optimization in a Hilbert spaceMathematical Programming, 1998
- A Probabilistic Theory of Pattern Recognition1996
- Statistical Decision Theory and Bayesian Analysis1985
- A Problem of Dimensionality: A Simple ExampleIEEE Transactions on Pattern Analysis and Machine Intelligence, 1979