Efficient Nonparametric Tensor Decomposition for Binary and Count Data
Preprint
Abstract
In numerous applications, binary reactions or event counts are observed and stored within high-order tensors. Tensor decompositions (TDs) serve as a powerful tool to handle such high-dimensional and sparse data. However, many traditional TDs are explicitly or implicitly designed based on the Gaussian distribution, which is unsuitable for discrete data. Moreover, most TDs rely on predefined multi-linear structures, such as CP and Tucker formats. Therefore, they may not be effective enough to handle complex real-world datasets. To address these issues, we propose ENTED, an \underline{E}fficient \underline{N}onparametric \underline{TE}nsor \underline{D}ecomposition for binary and count tensors. Specifically, we first employ a nonparametric Gaussian process (GP) to replace traditional multi-linear structures. Next, we utilize the \pg augmentation which provides a unified framework to establish conjugate models for binary and count distributions. Finally, to address the computational issue of GPs, we enhance the model by incorporating sparse orthogonal variational inference of inducing points, which offers a more effective covariance approximation within GPs and stochastic natural gradient updates for nonparametric models. We evaluate our model on several real-world tensor completion tasks, considering binary and count datasets. The results manifest both better performance and computational advantages of the proposed model.Keywords
All Related Versions
This publication has 7 references indexed in Scilit:
- An interactive web-based dashboard to track COVID-19 in real timeThe Lancet Infectious Diseases, 2020
- Generalized Canonical Polyadic Tensor DecompositionSIAM Review, 2020
- Tensor Networks for Dimensionality Reduction and Large-scale Optimization: Part 1 Low-Rank Tensor DecompositionsFoundations and Trends in Machine Learning, 2016
- On Tensors, Sparsity, and Nonnegative FactorizationsSIAM Journal on Matrix Analysis and Applications, 2012
- Tensor-Train DecompositionSIAM Journal on Scientific Computing, 2011
- Some Mathematical Notes on Three-Mode Factor AnalysisPsychometrika, 1966
- The Expression of a Tensor or a Polyadic as a Sum of ProductsJournal of Mathematics and Physics, 1927