Bayesian Additive Regression Trees with Model Trees
Authors:
- Estevão B. PradoEPEstevão B. Prado
- Rafael A. MoralRMRafael A. Moral
- Andrew C. ParnellAPAndrew C. Parnell
Preprint
Abstract
Bayesian Additive Regression Trees (BART) is a tree-based machine learning method that has been successfully applied to regression and classification problems. BART assumes regularisation priors on a set of trees that work as weak learners and is very flexible for predicting in the presence of non-linearity and high-order interactions. In this paper, we introduce an extension of BART, called Model Trees BART (MOTR-BART), that considers piecewise linear functions at node levels instead of piecewise constants. In MOTR-BART, rather than having a unique value at node level for the prediction, a linear predictor is estimated considering the covariates that have been used as the split variables in the corresponding tree. In our approach, local linearities are captured more efficiently and fewer trees are required to achieve equal or better performance than BART. Via simulation studies and real data applications, we compare MOTR-BART to its main competitors. R code for MOTR-BART implementation is available at https://github.com/ebprado/MOTR-BART.Keywords
All Related Versions
This publication has 28 references indexed in Scilit:
- Local Linear ForestsJournal of Computational and Graphical Statistics, 2020
- Heteroscedastic BART via Multiplicative Regression TreesJournal of Computational and Graphical Statistics, 2019
- Bayesian Regression Trees for High-Dimensional Prediction and Variable SelectionJournal of the American Statistical Association, 2018
- Bayesian Additive Regression Trees using Bayesian model averagingStatistics and Computing, 2017
- Subgroup finding via Bayesian additive regression treesStatistics in Medicine, 2017
- Nonparametric survival analysis using Bayesian Additive Regression Trees (BART)Statistics in Medicine, 2016
- Multinomial probit Bayesian additive regression treesStat, 2016
- Bayesian methods for proteomic biomarker developmentEuPA Open Proteomics, 2015
- The Bayesian Additive Classification Tree applied to credit risk modellingComputational Statistics & Data Analysis, 2010
- Logistic Model TreesMachine Learning, 2005