A Bayesian Nonparametric Learning Approach to Ensemble Models Using the Proper Bayesian Bootstrap
Open Access
- 3 January 2021
- journal article
- research article
- Published by MDPI AG in Algorithms
- Vol. 14 (1) , 11
Abstract
Bootstrap resampling techniques, introduced by Efron and Rubin, can be presented in a general Bayesian framework, approximating the statistical distribution of a statistical functional , where F is a random distribution function. Efron’s and Rubin’s bootstrap procedures can be extended, introducing an informative prior through the Proper Bayesian bootstrap. In this paper different bootstrap techniques are used and compared in predictive classification and regression models based on ensemble approaches, i.e., bagging models involving decision trees. Proper Bayesian bootstrap, proposed by Muliere and Secchi, is used to sample the posterior distribution over trees, introducing prior distributions on the covariates and the target variable. The results obtained are compared with respect to other competitive procedures employing different bootstrap techniques. The empirical analysis reports the results obtained on simulated and real data.
Keywords
All Related Versions
This publication has 20 references indexed in Scilit:
- Bayesian bootstrap predictionJournal of Statistical Planning and Inference, 2010
- Bayesians, Frequentists, and ScientistsJournal of the American Statistical Association, 2005
- Bayesian CART Model SearchJournal of the American Statistical Association, 1998
- Multivariate Adaptive Regression SplinesThe Annals of Statistics, 1991
- On a Second-Order Asymptotic Property of the Bayesian Bootstrap MeanThe Annals of Statistics, 1989
- A Large Sample Study of the Bayesian BootstrapThe Annals of Statistics, 1987
- The Bayesian BootstrapThe Annals of Statistics, 1981
- Mixtures of Dirichlet Processes with Applications to Bayesian Nonparametric ProblemsThe Annals of Statistics, 1974
- The jackknife-a reviewBiometrika, 1974
- A Bayesian Analysis of Some Nonparametric ProblemsThe Annals of Statistics, 1973