Symmetry Breaking and Training from Incomplete Data with Radial Basis Boltzmann Machines
- 1 June 1997
- journal article
- research article
- Published by World Scientific Pub Co Pte Ltd in International Journal of Neural Systems
- Vol. 08 (03) , 301-315
Abstract
A Radial Basis Boltzmann Machine (RBBM) is a specialized Boltzmann Machine architecture that combines feed-forward mapping with probability estimation in the input space, and for which very efficient learning rules exist. The hidden representation of the network displays symmetry breaking as a function of the noise in the dynamics. Thus, generalization can be studied as a function of the noise in the neuron dynamics instead of as a function of the number of hidden units. We show that the RBBM can be seen as an elegant alternative of k-nearest neighbor, leading to comparable performance without the need to store all data. We show that the RBBM has good classification performance compared to the MLP. The main advantage of the RBBM is that simultaneously with the input-output mapping, a model of the input space is obtained which can be used for learning with missing values. We derive learning rules for the case of incomplete data, and show that they perform better on incomplete data than the traditional learning rules on a 'repaired' data set.Keywords
This publication has 12 references indexed in Scilit:
- Deterministic learning rules for boltzmann machinesNeural Networks, 1995
- Pattern-Mixture Models for Multivariate Incomplete DataJournal of the American Statistical Association, 1993
- Constrained clustering as an optimization methodIEEE Transactions on Pattern Analysis and Machine Intelligence, 1993
- Vector quantization by deterministic annealingIEEE Transactions on Information Theory, 1992
- Handwritten Digit Recognition Using K Nearest-Neighbor, Radial-Basis Function, and Backpropagation Neural NetworksNeural Computation, 1991
- Back-propagation algorithm which varies the number of hidden unitsNeural Networks, 1991
- Statistical mechanics and phase transitions in clusteringPhysical Review Letters, 1990
- A Learning Algorithm for Boltzmann Machines*Cognitive Science, 1985
- An Algorithm for Vector Quantizer DesignIEEE Transactions on Communications, 1980
- Pattern Recognition with Partly Missing DataIEEE Transactions on Systems, Man, and Cybernetics, 1979