Generalized Bayesian likelihood-free inference
Authors:
Open Access
- 1 January 2024
- journal article
- research article
- Published by Institute of Mathematical Statistics in Electronic Journal of Statistics
- Vol. 18 (2) , 3628-3686
Abstract
Generalized Bayesian inference replaces the likelihood in the Bayesian posterior with the exponential of a loss function connecting parameter values and observations. As a loss function, it is possible to use Scoring Rules (SRs), which evaluate the match between the observation and the probabilistic model for given parameter values. In this work, we leverage this Scoring Rule posterior for Bayesian Likelihood-Free Inference (LFI). In LFI, we can sample from the model but not evaluate the likelihood; hence, we use the Energy and Kernel SRs in the SR posterior, as they admit unbiased empirical estimates. While traditional Pseudo-Marginal (PM) Markov Chain Monte Carlo (MCMC) can be applied to the SR posterior, it mixes poorly for concentrated targets, such as those obtained with many observations. As such, we propose to use Stochastic Gradient (SG) MCMC, which improves performance over PM-MCMC and scales to higher-dimensional setups as it is rejection-free. SG-MCMC requires differentiating the simulator model; we achieve this effortlessly by implementing the simulator models using automatic differentiation libraries. We compare SG-MCMC sampling for the SR posterior with related LFI approaches and find that the former scales to larger sample sizes and works well on the raw data, while other methods require determining suitable summary statistics. On a chaotic dynamical system from meteorology, our method even allows inferring the parameters of a neural network used to parametrize a part of the update equations.Keywords
All Related Versions
This publication has 51 references indexed in Scilit:
- NuZZ: Numerical Zig-Zag for general modelsStatistics and Computing, 2024
- Efficient and generalizable tuning strategies for stochastic gradient MCMCStatistics and Computing, 2023
- Automatic Zig-Zag sampling in practiceStatistics and Computing, 2022
- Focused Bayesian predictionJournal of Applied Econometrics, 2021
- Robust Bayesian synthetic likelihood via a semi-parametric approachStatistics and Computing, 2019
- Energy distanceWIREs Computational Statistics, 2015
- Approximate Bayesian computation with composite score functionsStatistics and Computing, 2015
- An adaptive sequential Monte Carlo method for approximate Bayesian computationStatistics and Computing, 2011
- The Gaussian rank correlation estimator: robustness propertiesStatistics and Computing, 2011
- Convergence of Probability Measures1999