Fast Nonnegative Matrix/Tensor Factorization Based on Low-Rank Approximation
- 8 March 2012
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Signal Processing
- Vol. 60 (6) , 2928-2940
Abstract
Nonnegative matrix factorization (NMF) algorithms often suffer from slow convergence speed due to the nonnegativity constraints, especially for large-scale problems. Low-rank approximation methods such as principle component analysis (PCA) are widely used in matrix factorizations to suppress noise, reduce computational complexity and memory requirements. However, they cannot be applied to NMF directly so far as they result in factors with mixed signs. In this paper, low-rank approximation is introduced to NMF (named lraNMF), which is not only able to reduce the computational complexity of NMF algorithms significantly, but also suppress bipolar noise. In fact, the new update rules are typically about times faster than traditional ones of NMF, here is the number of observations and is the low rank of latent factors. Therefore lraNMF is particularly efficient in the case where , which is the general case in NMF. The proposed update rules can also be incorporated into most existing NMF algorithms straightforwardly as long as they are based on Euclidean distance. Then the concept of lraNMF is generalized to the tensor field to perform a fast sequential nonnegative Tucker decomposition (NTD). By applying the proposed methods, the practicability of NMF/NTD is significantly improved. Simulations on synthetic and real data show the validity and efficiency of the proposed approaches.Keywords
This publication has 28 references indexed in Scilit:
- Extended HALS algorithm for nonnegative Tucker decomposition and its applications for multiway analysis and classificationNeurocomputing, 2011
- Generalizing the column–row matrix decomposition to multi-way arraysLinear Algebra and its Applications, 2010
- Non-Negative Matrix Factorization for Semisupervised Heterogeneous Data CoclusteringIEEE Transactions on Knowledge and Data Engineering, 2009
- Fast Local Algorithms for Large Scale Nonnegative Matrix and Tensor FactorizationsIEICE Transactions on Fundamentals of Electronics, Communications and Computer Sciences, 2009
- Toward Faster Nonnegative Matrix Factorization: A New Algorithm and Comparisons2008
- Convex and Semi-Nonnegative Matrix FactorizationsIEEE Transactions on Pattern Analysis and Machine Intelligence, 2008
- Algorithms for Sparse Nonnegative Tucker DecompositionsNeural Computation, 2008
- Nonnegative Matrix Factorization Based on Alternating Nonnegativity Constrained Least Squares and Active Set MethodSIAM Journal on Matrix Analysis and Applications, 2008
- Projected Gradient Methods for Nonnegative Matrix FactorizationNeural Computation, 2007
- Fast Monte Carlo Algorithms for Matrices II: Computing a Low-Rank Approximation to a MatrixSIAM Journal on Computing, 2006