Reversible Jump PDMP Samplers for Variable Selection
Open Access
- 7 September 2022
- journal article
- research article
- Published by Taylor & Francis in Journal of the American Statistical Association
- Vol. 118 (544) , 2915-2927
Abstract
A new class of Markov chain Monte Carlo (MCMC) algorithms, based on simulating piecewise deterministic Markov processes (PDMPs), has recently shown great promise: they are nonreversible, can mix better than standard MCMC algorithms, and can use subsampling ideas to speed up computation in big data scenarios. However, current PDMP samplers can only sample from posterior densities that are differentiable almost everywhere, which precludes their use for model choice. Motivated by variable selection problems, we show how to develop reversible jump PDMP samplers that can jointly explore the discrete space of models and the continuous space of parameters. Our framework is general: it takes any existing PDMP sampler, and adds two types of trans-dimensional moves that allow for the addition or removal of a variable from the model. We show how the rates of these trans-dimensional moves can be calculated so that the sampler has the correct invariant distribution. We remove a variable from a model when the associated parameter is zero, and this means that the rates of the trans-dimensional moves do not depend on the likelihood. It is, thus, easy to implement a reversible jump version of any PDMP sampler that can explore a fixed model. Supplementary materials for this article are available online.Keywords
All Related Versions
Funding Information
- EPSRC (EP/R018561, EP/R034710)
- Engineering and Physical Sciences Research Council
This publication has 34 references indexed in Scilit:
- MCMC Using Hamiltonian Dynamics2011
- The horseshoe estimator for sparse signalsBiometrika, 2010
- Spike and slab variable selection: Frequentist and Bayesian strategiesThe Annals of Statistics, 2005
- Analysis of a nonreversible Markov chain samplerThe Annals of Applied Probability, 2000
- Bayesian analysis of mixture models with an unknown number of components—an alternative to reversible jump methodsThe Annals of Statistics, 2000
- Nonparametric regression using Bayesian variable selectionJournal of Econometrics, 1996
- Reversible jump Markov chain Monte Carlo computation and Bayesian model determinationBiometrika, 1995
- Variable Selection via Gibbs SamplingJournal of the American Statistical Association, 1993
- Simulated Tempering: A New Monte Carlo SchemeEurophysics Letters, 1992
- Bayesian Variable Selection in Linear RegressionJournal of the American Statistical Association, 1988