Learning From Hidden Traits: Joint Factor Analysis and Latent Clustering
Open Access
- 29 September 2016
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Signal Processing
- Vol. 65 (1) , 256-269
Abstract
Dimensionality reduction techniques play an essential role in data analytics, signal processing, and machine learning. Dimensionality reduction is usually performed in a preprocessing stage that is separate from subsequent data analysis, such as clustering or classification. Finding reduced-dimension representations that are well-suited for the intended task is more appealing. This paper proposes a joint factor analysis and latent clustering framework, which aims at learning cluster-aware low-dimensional representations of matrix and tensor data. The proposed approach leverages matrix and tensor factorization models that produce essentially unique latent representations of the data to unravel latent cluster structure-which is otherwise obscured because of the freedom to apply an oblique transformation in latent space. At the same time, latent cluster structure is used as prior information to enhance the performance of factorization. Specific contributions include several custom-built problem formulations, corresponding algorithms, and discussion of associated convergence properties. Besides extensive simulations, real-world datasets such as Reuters document data and MNIST image data are also employed to showcase the effectiveness of the proposed approaches.Keywords
All Related Versions
Funding Information
- National Science Foundation (IIS-1447788, IIS-1247632)
This publication has 40 references indexed in Scilit:
- A Signal Processing Perspective on Hyperspectral Unmixing: Insights from Remote SensingIEEE Signal Processing Magazine, 2013
- A Block Coordinate Descent Method for Regularized Multiconvex Optimization with Applications to Nonnegative Tensor Factorization and CompletionSIAM Journal on Imaging Sciences, 2013
- Distributed Optimization and Statistical Learning via the Alternating Direction Method of MultipliersFoundations and Trends in Machine Learning, 2011
- ICE: a statistical approach to identifying endmembers in hyperspectral imagesIEEE Transactions on Geoscience and Remote Sensing, 2004
- On the uniqueness of multilinear decomposition ofN-way arraysJournal of Chemometrics, 2000
- Nearest q-Flat to m PointsJournal of Optimization Theory and Applications, 2000
- Probabilistic latent semantic indexing1999
- Metrics and models for handwritten character recognitionStatistical Science, 1998
- Algorithm AS 136: A K-Means Clustering AlgorithmJournal of the Royal Statistical Society Series C: Applied Statistics, 1979
- Three-way arrays: rank and uniqueness of trilinear decompositions, with application to arithmetic complexity and statisticsLinear Algebra and its Applications, 1977