Applying non-negative matrix factorization with covariates to label matrix for classification
Author:
Preprint
Abstract
Non-negative matrix factorization (NMF) is widely used for dimensionality reduction and interpretable analysis, but standard formulations are unsupervised and cannot directly exploit class labels. Existing supervised or semi-supervised extensions usually incorporate labels only via penalties or graph constraints, still requiring an external classifier. We propose \textit{NMF-LAB} (Non-negative Matrix Factorization for Label Matrix), which redefines classification as the inverse problem of non-negative matrix tri-factorization (tri-NMF). Unlike joint NMF methods, which reconstruct both features and labels, NMF-LAB directly factorizes the label matrix $Y$ as the observation, while covariates $A$ are treated as given explanatory variables. This yields a direct probabilistic mapping from covariates to labels, distinguishing our method from label-matrix factorization approaches that mainly model label correlations or impute missing labels. Our inversion offers two key advantages: (i) class-membership probabilities are obtained directly from the factorization without a separate classifier, and (ii) covariates, including kernel-based similarities, can be seamlessly integrated to generalize predictions to unseen samples. In addition, unlabeled data can be encoded as uniform distributions, supporting semi-supervised learning. Experiments on diverse datasets, from small-scale benchmarks to the large-scale MNIST dataset, demonstrate that NMF-LAB achieves competitive predictive accuracy, robustness to noisy or incomplete labels, and scalability to high-dimensional problems, while preserving interpretability. By unifying regression and classification within the tri-NMF framework, NMF-LAB provides a novel, probabilistic, and scalable approach to modern classification tasks.
Keywords
All Related Versions
This publication has 15 references indexed in Scilit:
- Applying non-negative matrix factorization with covariates to multivariate time series data as a vector autoregression modelJapanese Journal of Statistics and Data Science, 2025
- Initialization for non-negative matrix factorization: a comprehensive reviewInternational Journal of Data Science and Analytics, 2022
- Semi-supervised Nonnegative Matrix Factorization for Document Classification2021
- A novel general kernel-based non-negative matrix factorisation approach for face recognitionConnection Science, 2021
- Adaptive graph regularized nonnegative matrix factorization for data representationApplied Intelligence, 2019
- Supervised non-negative matrix factorization methods for MALDI imaging applicationsBioinformatics, 2018
- NMFE-SSCC: Non-negative matrix factorization ensemble for semi-supervised collective classificationKnowledge-Based Systems, 2015
- Linear and Nonlinear Projective Nonnegative Matrix FactorizationIEEE Transactions on Neural Networks, 2010
- Learning the parts of objects by non-negative matrix factorizationNature, 1999
- Some results on Tchebycheffian spline functionsJournal of Mathematical Analysis and Applications, 1971