Bayes and empirical Bayes: do they merge?
- 9 April 2014
- journal article
- research article
- Published by Oxford University Press (OUP) in Biometrika
- Vol. 101 (2) , 285-302
Abstract
Bayesian inference is attractive due to its internal coherence and for often having good frequentist properties. However, eliciting an honest prior may be difficult, and common practice is to take an empirical Bayes approach using an estimate of the prior hyperparameters. Although not rigorous, the underlying idea is that, for a sufficiently large sample size, empirical Bayes methods should lead to similar inferential answers as a proper Bayesian inference. However, precise mathematical results on this asymptotic agreement seem to be missing. In this paper, we give results in terms of merging Bayesian and empirical Bayes posterior distributions. We study two notions of merging: Bayesian weak merging and frequentist merging in total variation. We also show that, under regularity conditions, the empirical Bayes approach asymptotically gives an oracle selection of the prior hyperparameters. Examples include empirical Bayes density estimation with Dirichlet process mixtures.Keywords
All Related Versions
This publication has 25 references indexed in Scilit:
- The semiparametric Bernstein–von Mises theoremThe Annals of Statistics, 2012
- A semiparametric Bernstein–von Mises theorem for Gaussian process priorsProbability Theory and Related Fields, 2010
- Empirical Bayes vs. fully Bayes variable selectionJournal of Statistical Planning and Inference, 2008
- Empirical bayesian test of the smoothnessMathematical Methods of Statistics, 2008
- Flexible Empirical Bayes Estimation for WaveletsJournal of the Royal Statistical Society Series B: Statistical Methodology, 2000
- The consistency of posterior distributions in nonparametric problemsThe Annals of Statistics, 1999
- Choice of hierarchical priors: admissibility in estimation of normal meansThe Annals of Statistics, 1996
- Subjective Hierarchical Bayes Estimation of a Multivariate Normal Mean: On the Frequentist InterfaceThe Annals of Statistics, 1990
- On the Consistency of Bayes EstimatesThe Annals of Statistics, 1986
- Merging of Opinions with Increasing InformationThe Annals of Mathematical Statistics, 1962