Matrix Product Operator Restricted Boltzmann Machines
Preprint
Abstract
A restricted Boltzmann machine (RBM) learns a probability distribution over its input samples and has numerous uses like dimensionality reduction, classification and generative modeling. Conventional RBMs accept vectorized data that dismisses potentially important structural information in the original tensor (multi-way) input. Matrix-variate and tensor-variate RBMs, named MvRBM and TvRBM, have been proposed but are all restrictive by model construction, which leads to a weak model expression power. This work presents the matrix product operator RBM (MPORBM) that utilizes a tensor network generalization of Mv/TvRBM, preserves input formats in both the visible and hidden layers, and results in higher expressive power. A novel training algorithm integrating contrastive divergence and an alternating optimization procedure is also developed. Numerical experiments compare the MPORBM with the traditional RBM and MvRBM for data classification and image completion and denoising tasks. The expressive power of the MPORBM as a function of the MPO-rank is also investigated.Keywords
All Related Versions
This publication has 6 references indexed in Scilit:
- Unsupervised Generative Modeling Using Matrix Product StatesPhysical Review X, 2018
- Equivalence of restricted Boltzmann machines and tensor network statesPhysical Review B, 2018
- Tensor-Variate Restricted Boltzmann MachinesProceedings of the AAAI Conference on Artificial Intelligence, 2015
- A practical introduction to tensor networks: Matrix product states and projected entangled pair statesAnnals of Physics, 2014
- Tensor-Train DecompositionSIAM Journal on Scientific Computing, 2011
- Training Products of Experts by Minimizing Contrastive DivergenceNeural Computation, 2002