Automatic Relevance Determination in Nonnegative Matrix Factorization with the β-Divergence
Preprint
Abstract
This paper addresses the estimation of the latent dimensionality in nonnegative matrix factorization (NMF) with the \beta-divergence. The \beta-divergence is a family of cost functions that includes the squared Euclidean distance, Kullback-Leibler and Itakura-Saito divergences as special cases. Learning the model order is important as it is necessary to strike the right balance between data fidelity and overfitting. We propose a Bayesian model based on automatic relevance determination in which the columns of the dictionary matrix and the rows of the activation matrix are tied together through a common scale parameter in their prior. A family of majorization-minimization algorithms is proposed for maximum a posteriori (MAP) estimation. A subset of scale parameters is driven to a small lower bound in the course of inference, with the effect of pruning the corresponding spurious components. We demonstrate the efficacy and robustness of our algorithms by performing extensive experiments on synthetic data, the swimmer dataset, a music decomposition example and a stock price prediction task.Keywords
All Related Versions
This publication has 17 references indexed in Scilit:
- Automatic Relevance Determination in Nonnegative Matrix Factorization with the /spl beta/-DivergenceIEEE Transactions on Pattern Analysis and Machine Intelligence, 2012
- Unified Development of Multiplicative Algorithms for Linear and Quadratic Nonnegative Matrix FactorizationIEEE Transactions on Neural Networks, 2011
- Latent Variable Bayesian Models for Promoting SparsityIEEE Transactions on Information Theory, 2011
- Linear and Nonlinear Projective Nonnegative Matrix FactorizationIEEE Transactions on Neural Networks, 2010
- Automatic relevance determination for multi‐way modelsJournal of Chemometrics, 2009
- Enhancing Sparsity by Reweighted ℓ 1 MinimizationJournal of Fourier Analysis and Applications, 2008
- Improving molecular cancer class discovery through sparse non-negative matrix factorizationBioinformatics, 2005
- Learning the parts of objects by non-negative matrix factorizationNature, 1999
- Robust and efficient estimation by minimising a density power divergenceBiometrika, 1998
- Positive matrix factorization: A non‐negative factor model with optimal utilization of error estimates of data valuesEnvironmetrics, 1994