Regularized Gaussian Mixture Model for High-Dimensional Clustering
- 27 June 2018
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Cybernetics
- Vol. 49 (10) , 3677-3688
Abstract
Finding low-dimensional representation of high-dimensional data sets is an important task in various applications. The fact that data sets often contain clusters embedded in different subspaces poses barrier to this task. Driven by the need in methods that enable clustering and finding each cluster's intrinsic subspace simultaneously, in this paper, we propose a regularized Gaussian mixture model (GMM) for clustering. Despite the advantages of GMM, such as its probabilistic interpretation and robustness against observation noise, traditional maximum-likelihood estimation for GMMs shows disappointing performance in high-dimensional setting. The proposed regularization method finds low-dimensional representations of the component covariance matrices, resulting in better estimation of local feature correlations. The regularization problem can be incorporated in the expectation maximization algorithm for maximizing the likelihood function of a GMM, with the M-step modified to incorporate the regularization. The M-step involves a determinant maximization problem, which can be solved efficiently. The performance of the proposed method is demonstrated using several simulated data sets. We also illustrate the potential value of the proposed method in applications using four real data sets.Keywords
Funding Information
- National Natural Science Foundation of China (11471275)
- Research Grants Council Theme-based Research Scheme (T32-101/15-R, T32-102/14-N)
This publication has 49 references indexed in Scilit:
- A survey of multilinear subspace learning for tensor dataPattern Recognition, 2011
- PCA consistency in high dimension, low sample size contextThe Annals of Statistics, 2009
- On Consistency and Sparsity for Principal Components Analysis in High DimensionsJournal of the American Statistical Association, 2009
- Sparse inverse covariance estimation with the graphical lassoBiostatistics, 2007
- Automatic feature selection for unsupervised clustering of cycle-based signals in manufacturing processesIIE Transactions, 2005
- Using uncorrelated discriminant analysis for tissue classification with gene expression dataIEEE/ACM Transactions on Computational Biology and Bioinformatics, 2004
- Model-Based Clustering, Discriminant Analysis, and Density EstimationJournal of the American Statistical Association, 2002
- Speaker Verification Using Adapted Gaussian Mixture ModelsDigital Signal Processing, 2000
- Gaussian parsimonious clustering modelsPattern Recognition, 1995
- Algorithm AS 136: A K-Means Clustering AlgorithmJournal of the Royal Statistical Society Series C: Applied Statistics, 1979