TRUST-TECH-Based Expectation Maximization for Learning Finite Mixture Models
Authors:
- Chandan K. ReddyCRChandan K. Reddy
- Hsiao-Dong ChiangHCHsiao-Dong Chiang
- Bala RajaratnamBRBala Rajaratnam
- 6 June 2008
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Pattern Analysis and Machine Intelligence
- Vol. 30 (7) , 1146-1157
Abstract
The expectation maximization (EM) algorithm is widely used for learning finite mixture models despite its greedy nature. Most popular model-based clustering techniques might yield poor clusters if the parameters are not initialized properly. To reduce the sensitivity of initial points, a novel algorithm for learning mixture models from multivariate data is introduced in this paper. The proposed algorithm takes advantage of TRUST-TECH (TRansformation Under STability-reTaining Equilibria CHaracterization) to compute neighborhood local maxima on the likelihood surface using stability regions. Basically, our method coalesces the advantages of the traditional EM with that of the dynamic and geometric characteristics of the stability regions of the corresponding nonlinear dynamical system of the log-likelihood function. Two phases, namely, the EM phase and the stability region phase, are repeated alternatively in the parameter space to achieve local maxima with improved likelihood values. The EM phase obtains the local maximum of the likelihood function and the stability region phase helps to escape out of the local maximum by moving toward the neighboring stability regions. Though applied to Gaussian mixtures in this paper, our technique can be easily generalized to any other parametric finite mixture model. The algorithm has been tested on both synthetic and real data sets and the improvements in the performance compared to other approaches are demonstrated. The robustness with respect to initialization is also illustrated experimentally.Keywords
This publication has 33 references indexed in Scilit:
- A Stability Boundary Based Method for Finding Saddle Points on Potential Energy SurfacesJournal of Computational Biology, 2006
- Genetic-based EM algorithm for learning Gaussian mixture modelsIEEE Transactions on Pattern Analysis and Machine Intelligence, 2005
- Recursive unsupervised learning of finite mixture modelsIEEE Transactions on Pattern Analysis and Machine Intelligence, 2004
- Competitive EM algorithm for finite mixture modelsPattern Recognition, 2004
- Blobworld: image segmentation using expectation-maximization and its application to image queryingIEEE Transactions on Pattern Analysis and Machine Intelligence, 2002
- Minimum-entropy data partitioning using reversible jump Markov chain Monte CarloIEEE Transactions on Pattern Analysis and Machine Intelligence, 2001
- Text Classification from Labeled and Unlabeled Documents using EMMachine Learning, 2000
- A View of the Em Algorithm that Justifies Incremental, Sparse, and other Variants1998
- On Bayesian Analysis of Mixtures with an Unknown Number of Components (with discussion)Journal of the Royal Statistical Society Series B: Statistical Methodology, 1997
- Mixture Densities, Maximum Likelihood and the EM AlgorithmSIAM Review, 1984