Exact solvable deep Boltzmann machine
Open Access
- 1 January 2025
- journal article
- Published by Institute of Electronics, Information and Communications Engineers (IEICE) in Nonlinear Theory and Its Applications, IEICE
- Vol. 16 (3) , 410-421
Abstract
In training and inference in a deep Boltzmann machine (DBM), the evaluation of the free energy is crucial. Due to combinatorial explosion, its evaluation is computationally infeasible. We investigate the free energy evaluation of the DBM based on the Bethe free energy representation, which is related to the belief propagation algorithm. This representation significantly reduces the computational cost of the evaluation and enables the exact computation of the free energy in a slim DBM, i.e., a DBM with small-sized layers. Finally, we discuss the properties of the free energy in slim DBMs based on numerical and analytical methods.Keywords
This publication has 8 references indexed in Scilit:
- Deep Medical Image Reconstruction with Autoencoders using Deep Boltzmann Machine TrainingEAI Endorsed Transactions on Pervasive Health and Technology, 2020
- Constructing exact representations of quantum many-body systems with deep neural networksNature Communications, 2018
- Deep-FS: A feature selection algorithm for Deep Boltzmann MachinesNeurocomputing, 2018
- A Joint Deep Boltzmann Machine (jDBM) Model for Person Identification Using Mobile Phone DataIEEE Transactions on Multimedia, 2016
- Classification using discriminative restricted Boltzmann machines2008
- Cluster variation method in statistical physics and probabilistic graphical modelsJournal of Physics A: General Physics, 2005
- Constructing Free-Energy Approximations and Generalized Belief Propagation AlgorithmsIEEE Transactions on Information Theory, 2005
- Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference.The Journal of Philosophy, 1991