Spike-and-Slab Group Lassos for Grouped Regression and Sparse Generalized Additive Models
Preprint
Abstract
We introduce the spike-and-slab group lasso (SSGL) for Bayesian estimation and variable selection in linear regression with grouped variables. We further extend the SSGL to sparse generalized additive models (GAMs), thereby introducing the first nonparametric variant of the spike-and-slab lasso methodology. Our model simultaneously performs group selection and estimation, while our fully Bayes treatment of the mixture proportion allows for model complexity control and automatic self-adaptivity to different levels of sparsity. We develop theory to uniquely characterize the global posterior mode under the SSGL and introduce highly efficient coordinate ascent algorithms for maximum a posteriori (MAP) estimation. We further employ de-biasing methods to provide uncertainty quantification of our MAP estimates. Thus, implementation of our model avoids the computational intensiveness of Markov chain Monte Carlo (MCMC) in high dimensions. We derive posterior concentration rates for both grouped linear regression and sparse GAMs when the number of covariates grows at nearly exponential rate with sample size. Finally, we illustrate our methodology through extensive simulations and data analysis.Keywords
All Related Versions
This publication has 31 references indexed in Scilit:
- Simultaneous Variable and Covariance Selection With the Multivariate Spike-and-Slab LASSOJournal of Computational and Graphical Statistics, 2019
- Bayesian Regularization for Graphical Models With Unequal ShrinkageJournal of the American Statistical Association, 2018
- Group spike-and-slab lasso generalized linear models for disease prediction and associated genes detection by incorporating pathway informationBioinformatics, 2017
- The spike-and-slab lasso Cox model for survival prediction and associated genes detectionBioinformatics, 2017
- Bayesian Sparse Group SelectionJournal of Computational and Graphical Statistics, 2016
- Dirichlet–Laplace Priors for Optimal ShrinkageJournal of the American Statistical Association, 2015
- Multivariate Sparse Group Lasso for the Multivariate Multiple Linear Regression with an Arbitrary Group StructureBiometrics, 2015
- A Sparse-Group LassoJournal of Computational and Graphical Statistics, 2013
- Sparse Additive ModelsJournal of the Royal Statistical Society Series B: Statistical Methodology, 2009
- Model Selection and Estimation in Regression with Grouped VariablesJournal of the Royal Statistical Society Series B: Statistical Methodology, 2005