Mixture Models With a Prior on the Number of Components
- 13 November 2017
- journal article
- research article
- Published by Taylor & Francis in Journal of the American Statistical Association
- Vol. 113 (521) , 340-356
Abstract
A natural Bayesian approach for mixture models with an unknown number of components is to take the usual finite mixture model with symmetric Dirichlet weights, and put a prior on the number of components—that is, to use a mixture of finite mixtures (MFM). The most commonly used method of inference for MFMs is reversible jump Markov chain Monte Carlo, but it can be nontrivial to design good reversible jump moves, especially in high-dimensional spaces. Meanwhile, there are samplers for Dirichlet process mixture (DPM) models that are relatively simple and are easily adapted to new applications. It turns out that, in fact, many of the essential properties of DPMs are also exhibited by MFMs—an exchangeable partition distribution, restaurant process, random measure representation, and stick-breaking representation—and crucially, the MFM analogues are simple enough that they can be used much like the corresponding DPM properties. Consequently, many of the powerful methods developed for inference in DPMs can be directly applied to MFMs as well; this simplifies the implementation of MFMs and can substantially improve mixing. We illustrate with real and simulated data, including high-dimensional gene expression data used to discriminate cancer subtypes. Supplementary materials for this article are available online.Keywords
All Related Versions
This publication has 82 references indexed in Scilit:
- A Product Partition Model With Regression on CovariatesJournal of Computational and Graphical Statistics, 2011
- Random partition models with regression on covariatesJournal of Statistical Planning and Inference, 2010
- Clustering cancer gene expression data: a comparative studyBMC Bioinformatics, 2008
- Kernel stick-breaking processesBiometrika, 2008
- Estimating Mixture of Dirichlet Process ModelsJournal of Computational and Graphical Statistics, 1998
- Markov Chain Monte Carlo in Practice: A Roundtable DiscussionThe American Statistician, 1998
- Reversible jump Markov chain Monte Carlo computation and Bayesian model determinationBiometrika, 1995
- Bayesian Density Estimation and Inference Using MixturesJournal of the American Statistical Association, 1995
- The Collapsed Gibbs Sampler in Bayesian Computations with Applications to a Gene Regulation ProblemJournal of the American Statistical Association, 1994
- Density Estimation with Confidence Sets Exemplified by Superclusters and Voids in the GalaxiesJournal of the American Statistical Association, 1990