Learning with hidden variables
Authors:
Preprint
Abstract
Learning and inferring features that generate sensory input is a task continuously performed by cortex. In recent years, novel algorithms and learning rules have been proposed that allow neural network models to learn such features from natural images, written text, audio signals, etc. These networks usually involve deep architectures with many layers of hidden neurons. Here we review recent advancements in this area emphasizing, amongst other things, the processing of dynamical inputs by networks with hidden nodes and the role of single neuron models. These points and the questions they arise can provide conceptual advancements in understanding of learning in the cortex and the relationship between machine learning approaches to learning with hidden nodes and those in cortical circuits.Keywords
All Related Versions
This publication has 32 references indexed in Scilit:
- Belief propagation and replicas for inference and learning in a kinetic Ising model with hidden spinsJournal of Statistical Mechanics: Theory and Experiment, 2015
- Inferring hidden states in a random kinetic Ising model: replica analysisJournal of Statistical Mechanics: Theory and Experiment, 2014
- Exact mean-field inference in asymmetric kinetic Ising systemsJournal of Statistical Mechanics: Theory and Experiment, 2011
- Are spin-glass effects relevant to understanding realistic auto-associative networks?Journal of Physics A: General Physics, 1991
- Graded-response neurons and information encodings in autoassociative memoriesPhysical Review A, 1990
- Spin Glass Theory and BeyondPhysics Today, 1988
- Neural networks and physical systems with emergent collective computational abilities.Proceedings of the National Academy of Sciences of the United States of America, 1982
- Neocognitron: A self-organizing neural network model for a mechanism of pattern recognition unaffected by shift in positionBiological Cybernetics, 1980
- Representation and recognition of the spatial organization of three-dimensional shapesProceedings of the Royal Society of London. B. Biological Sciences, 1978
- Some informational aspects of visual perception.Psychological Review, 1954