Automatic Relevance Determination in Nonnegative Matrix Factorization with the /spl beta/-Divergence
- 12 November 2012
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Pattern Analysis and Machine Intelligence
- Vol. 35 (7) , 1592-1605
Abstract
This paper addresses the estimation of the latent dimensionality in nonnegative matrix factorization (NMF) with the β-divergence. The β-divergence is a family of cost functions that includes the squared euclidean distance, Kullback-Leibler (KL) and Itakura-Saito (IS) divergences as special cases. Learning the model order is important as it is necessary to strike the right balance between data fidelity and overfitting. We propose a Bayesian model based on automatic relevance determination (ARD) in which the columns of the dictionary matrix and the rows of the activation matrix are tied together through a common scale parameter in their prior. A family of majorization-minimization (MM) algorithms is proposed for maximum a posteriori (MAP) estimation. A subset of scale parameters is driven to a small lower bound in the course of inference, with the effect of pruning the corresponding spurious components. We demonstrate the efficacy and robustness of our algorithms by performing extensive experiments on synthetic data, the swimmer dataset, a music decomposition example, and a stock price prediction task.Keywords
All Related Versions
This publication has 24 references indexed in Scilit:
- Optimization with Sparsity-Inducing PenaltiesFoundations and Trends in Machine Learning, 2012
- Unified Development of Multiplicative Algorithms for Linear and Quadratic Nonnegative Matrix FactorizationIEEE Transactions on Neural Networks, 2011
- Latent Variable Bayesian Models for Promoting SparsityIEEE Transactions on Information Theory, 2011
- Generalized Alpha-Beta Divergences and Their Application to Robust Nonnegative Matrix FactorizationEntropy, 2011
- Linear and Nonlinear Projective Nonnegative Matrix FactorizationIEEE Transactions on Neural Networks, 2010
- Enhancing Sparsity by Reweighted ℓ 1 MinimizationJournal of Fourier Analysis and Applications, 2008
- Model Selection and Estimation in Regression with Grouped VariablesJournal of the Royal Statistical Society Series B: Statistical Methodology, 2005
- Sparse coding and NMF2005
- Probable networks and plausible predictions — a review of practical Bayesian methods for supervised neural networksNetwork: Computation in Neural Systems, 1995
- Estimating the Dimension of a ModelThe Annals of Statistics, 1978