BoostTree and BoostForest for Ensemble Learning
- 7 December 2022
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Pattern Analysis and Machine Intelligence
- Vol. PP (01628828) , 1-17
Abstract
Bootstrap aggregating (Bagging) and boosting are two popular ensemble learning approaches, which combine multiple base learners to generate a composite model for more accurate and more reliable performance. They have been widely used in biology, engineering, healthcare, etc. This paper proposes BoostForest, which is an ensemble learning approach using BoostTree as base learners and can be used for both classification and regression. BoostTree constructs a tree model by gradient boosting. It increases the randomness (diversity) by drawing the cut-points randomly at node splitting. BoostForest further increases the randomness by bootstrapping the training data in constructing different BoostTrees. BoostForest generally outperformed four classical ensemble learning approaches (Random Forest, Extra-Trees, XGBoost and LightGBM) on 35 classification and regression datasets. Remarkably, BoostForest tunes its parameters by simply sampling them randomly from a parameter pool, which can be easily specified, and its ensemble learning framework can also be used to combine many other base learners.Keywords
All Related Versions
This publication has 33 references indexed in Scilit:
- Massive computational acceleration by using neural networks to emulate mechanism-based biological modelsNature Communications, 2019
- Deep forestNational Science Review, 2018
- Deep Convolutional Neural Networks for Breast Cancer Histology Image Analysis2018
- Combining Partial Least Squares and the Gradient-Boosting Method for Soil Property Retrieval Using Visible Near-Infrared Shortwave Infrared SpectraRemote Sensing, 2017
- XGBoost2016
- Pedestrian Detection with Spatially Pooled Features and Structured Ensemble LearningIEEE Transactions on Pattern Analysis and Machine Intelligence, 2015
- Extreme learning machine: Theory and applicationsNeurocomputing, 2006
- An analysis of diversity measuresMachine Learning, 2006
- Extremely randomized treesMachine Learning, 2006
- Bagging predictorsMachine Learning, 1996