Variational Bayesian inference for models with nuisance parameters and an intractable likelihood
Open Access
- 17 June 2025
- journal article
- research article
- Published by Springer Nature in Statistics and Computing
- Vol. 35 (5) , 1-14
Abstract
A primary challenge in Bayesian analysis lies in computing the posterior distribution of model parameters, especially for models with a large number of parameters or intractable likelihoods. Often, the focus is on a subset of parameters, with the remainder regarded as nuisance parameters introduced for computational convenience. This complexity necessitates refined computational methods. Variational Bayesian inference (VB) has emerged as a powerful solution, enhancing computational efficiency by recasting inference as an optimization problem within a family of tractable distributions. However, common VB techniques sometimes fall short, especially for models with nuisance parameters or intractable likelihoods. After identifying characteristics of suboptimal VB methods, we build upon the Hybrid Variational Bayes (HVB) approach introduced by Loaiza-Maya et al. (2022) and develop an extended and unified HVB framework designed to achieve more precise Bayesian inference in such scenarios. Through theoretical exploration and a series of illustrative examples, our approach demonstrates notable improvements over traditional VB methods.Keywords
All Related Versions
Funding Information
- University of Sydney
This publication has 23 references indexed in Scilit:
- Fast and accurate variational inference for models with many latent variablesJournal of Econometrics, 2022
- Gaussian Variational Approximation With a Factor Covariance StructureJournal of Computational and Graphical Statistics, 2018
- Variational Bayes With Intractable LikelihoodJournal of Computational and Graphical Statistics, 2017
- Pseudo-marginal Metropolis–Hastings sampling using averages of unbiased estimatorsBiometrika, 2017
- Variational Inference: A Review for StatisticiansJournal of the American Statistical Association, 2017
- Generalized Linear Mixed Models2014
- On some properties of Markov chain Monte Carlo simulation methods based on the particle filterJournal of Econometrics, 2012
- Data Analysis Using Regression and Multilevel/Hierarchical Models2006
- An Introduction to Variational Methods for Graphical ModelsMachine Learning, 1999
- The Monte Carlo MethodJournal of the American Statistical Association, 1949