Adaptive Ensemble of Classifiers with Regularization for Imbalanced Data Classification
Authors:
Preprint
Abstract
The dynamic ensemble selection of classifiers is an effective approach for processing label-imbalanced data classifications. However, such a technique is prone to overfitting, owing to the lack of regularization methods and the dependence of the aforementioned technique on local geometry. In this study, focusing on binary imbalanced data classification, a novel dynamic ensemble method, namely adaptive ensemble of classifiers with regularization (AER), is proposed, to overcome the stated limitations. The method solves the overfitting problem through implicit regularization. Specifically, it leverages the properties of stochastic gradient descent to obtain the solution with the minimum norm, thereby achieving regularization; furthermore, it interpolates the ensemble weights by exploiting the global geometry of data to further prevent overfitting. According to our theoretical proofs, the seemingly complicated AER paradigm, in addition to its regularization capabilities, can actually reduce the asymptotic time and memory complexities of several other algorithms. We evaluate the proposed AER method on seven benchmark imbalanced datasets from the UCI machine learning repository and one artificially generated GMM-based dataset with five variations. The results show that the proposed algorithm outperforms the major existing algorithms based on multiple metrics in most cases, and two hypothesis tests (McNemar's and Wilcoxon tests) verify the statistical significance further. In addition, the proposed method has other preferred properties such as special advantages in dealing with highly imbalanced data, and it pioneers the research on the regularization for dynamic ensemble methods.Keywords
All Related Versions
This publication has 32 references indexed in Scilit:
- Machine learning based mobile malware detection using highly imbalanced network trafficInformation Sciences, 2018
- Learning from class-imbalanced data: Review of methods and applicationsExpert Systems with Applications, 2017
- Learning from imbalanced data: open challenges and future directionsProgress in Artificial Intelligence, 2016
- Evolutionary undersampling boosting for imbalanced classification of breast cancer malignancyApplied Soft Computing, 2016
- ROSEFW-RF: The winner algorithm for the ECBDL’14 big data competition: An extremely imbalanced big data bioinformatics problemKnowledge-Based Systems, 2015
- ELBlocker: Predicting blocking bugs with ensemble imbalance learningInformation and Software Technology, 2015
- A survey of multiple classifier systems as hybrid systemsInformation Fusion, 2014
- LibD3C: Ensemble classifiers with a clustering and dynamic selection strategyNeurocomputing, 2014
- Analysing the classification of imbalanced data-sets with multiple classes: Binarization techniques and ad-hoc approachesKnowledge-Based Systems, 2013
- Dynamic classifier ensemble model for customer classification with imbalanced class distributionExpert Systems with Applications, 2012