Resampling or Reweighting: A Comparison of Boosting Implementations
Authors:
- Chris SeiffertCSChris Seiffert
- Taghi M. KhoshgoftaarTKTaghi M. Khoshgoftaar
- Jason Van HulseJHJason Van Hulse
- Amri NapolitanoANAmri Napolitano
- 1 November 2008
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- Vol. 1 (10823409) , 445-451
Abstract
Boosting has been shown to improve the performance of classifiers in many situations, including when data is imbalanced. There are, however, two possible implementations of boosting, and it is unclear which should be used. Boosting by reweighting is typically used, but can only be applied to base learners which are designed to handle example weights. On the other hand, boosting by resampling can be applied to any base learner. In this work, we empirically evaluate the differences between these two boosting implementations using imbalanced training data. Using 10 boosting algorithms, 4 learners and 15 datasets, we find that boosting by resampling performs as well as, or significantly better than, boosting by reweighting (which is often the default boosting implementation). We therefore conclude that in general, boosting by resampling is preferred over boosting by weighting.Keywords
This publication has 12 references indexed in Scilit:
- Cost-sensitive boosting for classification of imbalanced dataPattern Recognition, 2007
- Learning with limited minority class data2007
- The relationship between Precision-Recall and ROC curves2006
- Precision-recall operating characteristic (P-ROC) curves in imprecise environments2006
- Mining with rarityACM SIGKDD Explorations Newsletter, 2004
- Evaluating boosting algorithms to classify rare classes: comparison and improvements2002
- Resampling vs Reweighting in Boosting a Relational Weak Learner2001
- Robust Classification for Imprecise EnvironmentsMachine Learning, 2001
- Lazy Learning1997
- Fast Effective Rule Induction1995