Bayesian Model Selection with Graph Structured Sparsity
Authors:
Preprint
Abstract
We propose a general algorithmic framework for Bayesian model selection. A spike-and-slab Laplacian prior is introduced to model the underlying structural assumption. Using the notion of effective resistance, we derive an EM-type algorithm with closed-form iterations to efficiently explore possible candidates for Bayesian model selection. The deterministic nature of the proposed algorithm makes it more scalable to large-scale and high-dimensional data sets compared with existing stochastic search algorithms. When applied to sparse linear regression, our framework recovers the EMVS algorithm [Rockova and George, 2014] as a special case. We also discuss extensions of our framework using tools from graph algebra to incorporate complex Bayesian models such as biclustering and submatrix localization. Extensive simulation studies and real data applications are conducted to demonstrate the superior performance of our methods over its frequentist competitors such as $\ell_0$ or $\ell_1$ penalization.
Keywords
All Related Versions
This publication has 18 references indexed in Scilit:
- A Dual Active-Set Algorithm for Regularized Monotonic RegressionJournal of Optimization Theory and Applications, 2017
- Volume Ratio, Sparsity, and Minimaxity Under Unitarily Invariant NormsIEEE Transactions on Information Theory, 2015
- EMVS: The EM Approach to Bayesian Variable SelectionJournal of the American Statistical Association, 2014
- Topics in Random Matrix Theory2012
- Robust biclustering by sparse singular value decomposition incorporating stability selectionBioinformatics, 2011
- Biclustering via Sparse Singular Value DecompositionBiometrics, 2010
- Sparsity and Smoothness Via the Fused LassoJournal of the Royal Statistical Society Series B: Statistical Methodology, 2004
- Classification of human lung carcinomas by mRNA expression profiling reveals distinct adenocarcinoma subclassesProceedings of the National Academy of Sciences, 2001
- Isotonic regression: Another look at the changepoint problemBiometrika, 2001
- On Bayesian Analysis of Mixtures with an Unknown Number of Components (with discussion)Journal of the Royal Statistical Society Series B: Statistical Methodology, 1997