Scalable Bayesian Variable Selection Using Nonlocal Prior Densities in Ultrahigh-Dimensional Settings
Preprint
Abstract
Bayesian model selection procedures based on nonlocal alternative prior densities are extended to ultrahigh dimensional settings and compared to other variable selection procedures using precision-recall curves. Variable selection procedures included in these comparisons include methods based on $g$-priors, reciprocal lasso, adaptive lasso, scad, and minimax concave penalty criteria. The use of precision-recall curves eliminates the sensitivity of our conclusions to the choice of tuning parameters. We find that Bayesian variable selection procedures based on nonlocal priors are competitive to all other procedures in a range of simulation scenarios, and we subsequently explain this favorable performance through a theoretical examination of their consistency properties. When certain regularity conditions apply, we demonstrate that the nonlocal procedures are consistent for linear models even when the number of covariates $p$ increases sub-exponentially with the sample size $n$. A model selection procedure based on Zellner's $g$-prior is also found to be competitive with penalized likelihood methods in identifying the true model, but the posterior distribution on the model space induced by this method is much more dispersed than the posterior distribution induced on the model space by the nonlocal prior methods. We investigate the asymptotic form of the marginal likelihood based on the nonlocal priors and show that it attains a unique term that cannot be derived from the other Bayesian model selection procedures. We also propose a scalable and efficient algorithm called Simplified Shotgun Stochastic Search with Screening (S5) to explore the enormous model space, and we show that S5 dramatically reduces the computing time without losing the capacity to search the interesting region in the model space. The S5 algorithm is available in an \verb R ~package {\it BayesS5} on \texttt{CRAN}.
Keywords
All Related Versions
This publication has 26 references indexed in Scilit:
- Nonlocal Priors for High-Dimensional EstimationJournal of the American Statistical Association, 2017
- Bayesian variable selection for binary outcomes in high-dimensional genomic studies using non-local priorsBioinformatics, 2016
- High-Dimensional Variable Selection With Reciprocal L1-RegularizationJournal of the American Statistical Association, 2015
- EMVS: The EM Approach to Bayesian Variable SelectionJournal of the American Statistical Association, 2014
- Bayesian Subset Modeling for High-Dimensional Generalized Linear ModelsJournal of the American Statistical Association, 2013
- Consistent High-Dimensional Bayesian Variable Selection via Penalized Credible RegionsJournal of the American Statistical Association, 2012
- Bayesian Model Selection in High-Dimensional SettingsJournal of the American Statistical Association, 2012
- Sure Independence Screening for Ultrahigh Dimensional Feature SpaceJournal of the Royal Statistical Society Series B: Statistical Methodology, 2008
- Extended Bayesian information criteria for model selection with large model spacesBiometrika, 2008
- Calibration and empirical Bayes variable selectionBiometrika, 2000