An Extension of Gradient Boosted Decision Tree Incorporating Statistical Tests
Authors:
- Ryuji SakataRSRyuji Sakata
- Iku OhamaIOIku Ohama
- Tadahiro TaniguchiTTTadahiro Taniguchi
- 1 November 2018
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 964-969
Abstract
Gradient boosted decision tree (GBDT) is one of the most popular machine learning methods for many machine learning and data mining tasks. In this paper, we address the overfitting problem of the existing GBDT algorithms, and introduce the idea of statistical significance into tree construction algorithm to reduce it. We propose a new algorithm, W-GBDT, incorporating Welch's t-test as a tree splitting criteria based on the existing XGBoost algorithm. Our experiment results, using real-world datasets, show that our proposed method significantly outperforms the original XGBoost in both the generalization ability and robustness against the number of iterations.Keywords
This publication has 11 references indexed in Scilit:
- XGBoost2016
- Feedback Prediction for Blogs2013
- Gradient boosting machines, a tutorialFrontiers in Neurorobotics, 2013
- Early Stopping — But When?2012
- Modeling wine preferences by data mining from physicochemical propertiesDecision Support Systems, 2009
- Avoiding Boosting Overfitting by Removing Confusing Samples2007
- Boosting with early stopping: Convergence and consistencyThe Annals of Statistics, 2005
- Greedy function approximation: A gradient boosting machine.The Annals of Statistics, 2001
- Modeling of strength of high-performance concrete using artificial neural networksCement and Concrete Research, 1998
- The Significance of the Difference Between Two Means when the Population Variances are UnequalBiometrika, 1938