Scalable Bayesian Tensor Ring Factorization for Multiway Data Analysis
Preprint
Abstract
Tensor decompositions play a crucial role in numerous applications related to multi-way data analysis. By employing a Bayesian framework with sparsity-inducing priors, Bayesian Tensor Ring (BTR) factorization offers probabilistic estimates and an effective approach for automatically adapting the tensor ring rank during the learning process. However, previous BTR method employs an Automatic Relevance Determination (ARD) prior, which can lead to sub-optimal solutions. Besides, it solely focuses on continuous data, whereas many applications involve discrete data. More importantly, it relies on the Coordinate-Ascent Variational Inference (CAVI) algorithm, which is inadequate for handling large tensors with extensive observations. These limitations greatly limit its application scales and scopes, making it suitable only for small-scale problems, such as image/video completion. To address these issues, we propose a novel BTR model that incorporates a nonparametric Multiplicative Gamma Process (MGP) prior, known for its superior accuracy in identifying latent structures. To handle discrete data, we introduce the Pólya-Gamma augmentation for closed-form updates. Furthermore, we develop an efficient Gibbs sampler for consistent posterior simulation, which reduces the computational complexity of previous VI algorithm by two orders, and an online EM algorithm that is scalable to extremely large tensors. To showcase the advantages of our model, we conduct extensive experiments on both simulation data and real-world applications.Keywords
All Related Versions
This publication has 11 references indexed in Scilit:
- Towards Flexible Sparsity-Aware Modeling: Automatic Tensor Rank Learning Using the Generalized Hyperbolic PriorIEEE Transactions on Signal Processing, 2022
- Bayesian Low Rank Tensor Ring for Image RecoveryIEEE Transactions on Image Processing, 2021
- Scaling Probabilistic Tensor Canonical Polyadic Decomposition to Massive DataIEEE Transactions on Signal Processing, 2018
- Bayesian CP Factorization of Incomplete Tensors with Automatic Rank DeterminationIEEE Transactions on Pattern Analysis and Machine Intelligence, 2015
- Sparse Bayesian infinite factor modelsBiometrika, 2011
- Scalable tensor factorizations for incomplete dataChemometrics and Intelligent Laboratory Systems, 2011
- Tensor-Train DecompositionSIAM Journal on Scientific Computing, 2011
- An Introduction to MCMC for Machine LearningMachine Learning, 2003
- Some Mathematical Notes on Three-Mode Factor AnalysisPsychometrika, 1966
- The Expression of a Tensor or a Polyadic as a Sum of ProductsJournal of Mathematics and Physics, 1927