How vague is vague? A simulation study of the impact of the use of vague prior distributions in MCMC using WinBUGS
- 15 August 2005
- journal article
- research article
- Published by Wiley in Statistics in Medicine
- Vol. 24 (15) , 2401-2428
Abstract
There has been a recent growth in the use of Bayesian methods in medical research. The main reasons for this are the development of computer intensive simulation based methods such as Markov chain Monte Carlo (MCMC), increases in computing power and the introduction of powerful software such as WinBUGS. This has enabled increasingly complex models to be fitted. The ability to fit these complex models has led to MCMC methods being used as a convenient tool by frequentists, who may have no desire to be fully Bayesian. Often researchers want ‘the data to dominate’ when there is no prior information and thus attempt to use vague prior distributions. However, with small amounts of data the use of vague priors can be problematic. The results are potentially sensitive to the choice of prior distribution. In general there are fewer problems with location parameters. The main problem is with scale parameters. With scale parameters, not only does one have to decide the distributional form of the prior distribution, but also whether to put the prior distribution on the variance, standard deviation or precision. We have conducted a simulation study comparing the effects of 13 different prior distributions for the scale parameter on simulated random effects meta‐analysis data. We varied the number of studies (5, 10 and 30) and compared three different between‐study variances to give nine different simulation scenarios. One thousand data sets were generated for each scenario and each data set was analysed using the 13 different prior distributions. The frequentist properties of bias and coverage were investigated for the between‐study variance and the effect size. The choice of prior distribution was crucial when there were just five studies. There was a large variation in the estimates of the between‐study variance for the 13 different prior distributions. With a large number of studies the choice of prior distribution was less important. The effect size estimated was not biased, but the precision with which it was estimated varied with the choice of prior distribution leading to varying coverage intervals and, potentially, to different statistical inferences. Again there was less of a problem with a larger number of studies. There is a particular problem if the between‐study variance is close to the boundary at zero, as MCMC results tend to produce upwardly biased estimates of the between‐study variance, particularly if inferences are based on the posterior mean. The choice of ‘vague’ prior distribution can lead to a marked variation in results, particularly in small studies. Sensitivity to the choice of prior distribution should always be assessed. Copyright © 2005 John Wiley & Sons, Ltd.Keywords
This publication has 28 references indexed in Scilit:
- Reference Bayesian Methods for Generalized Linear Mixed ModelsJournal of the American Statistical Association, 2000
- A prior for the variance in hierarchical modelsCanadian Journal of Statistics, 1999
- Markov chain Monte Carlo method and its applicationJournal of the Royal Statistical Society: Series D (The Statistician), 1998
- The Selection of Prior Distributions by Formal RulesJournal of the American Statistical Association, 1996
- Analysis of Clinical Data Using Imprecise Prior ProbabilitiesJournal of the Royal Statistical Society: Series D (The Statistician), 1996
- Bayesian approaches to random‐effects meta‐analysis: A comparative studyStatistics in Medicine, 1995
- Efficient parametrisations for normal linear mixed modelsBiometrika, 1995
- Reporting bayesian analyses of clinical trialsStatistics in Medicine, 1993
- Generalized Linear Models with Random Effects; a Gibbs Sampling ApproachJournal of the American Statistical Association, 1991
- The Combination of Estimates from Different ExperimentsBiometrics, 1954