Unsupervised Learning from Continuous Video in a Scalable Predictive Recurrent Network
Preprint
Abstract
Understanding visual reality involves acquiring common-sense knowledge about countless regularities in the visual world, e.g., how illumination alters the appearance of objects in a scene, and how motion changes their apparent spatial relationship. These regularities are hard to label for training supervised machine learning algorithms; consequently, algorithms need to learn these regularities from the real world in an unsupervised way. We present a novel network meta-architecture that can learn world dynamics from raw, continuous video. The components of this network can be implemented using any algorithm that possesses three key capabilities: prediction of a signal over time, reduction of signal dimensionality (compression), and the ability to use supplementary contextual information to inform the prediction. The presented architecture is highly-parallelized and scalable, and is implemented using localized connectivity, processing, and learning. We demonstrate an implementation of this architecture where the components are built from multi-layer perceptrons. We apply the implementation to create a system capable of stable and robust visual tracking of objects as seen by a moving camera. Results show performance on par with or exceeding state-of-the-art tracking algorithms. The tracker can be trained in either fully supervised or unsupervised-then-briefly-supervised regimes. Success of the briefly-supervised regime suggests that the unsupervised portion of the model extracts useful information about visual reality. The results suggest a new class of AI algorithms that uniquely combine prediction and scalability in a way that makes them suitable for learning from and --- and eventually acting within --- the real world.Keywords
All Related Versions
This publication has 35 references indexed in Scilit:
- What Happened at the DARPA Robotics Challenge Finals2018
- Reconstruction and Simulation of Neocortical MicrocircuitryCell, 2015
- Whatever next? Predictive brains, situated agents, and the future of cognitive scienceBehavioral and Brain Sciences, 2013
- Recurrent neuronal circuits in the neocortexCurrent Biology, 2007
- Contribution of feedforward, lateral and feedback connections to the classical receptive field center and extra-classical receptive field surround of primate V1 neurons2006
- Interpreting hippocampal function as recoding and forecastingNeural Networks, 2005
- Hierarchical Neural Networks for Image Interpretation2003
- Multilayer feedforward networks are universal approximatorsNeural Networks, 1989
- Generalization of backpropagation with application to a recurrent gas market modelNeural Networks, 1988
- Neocognitron: A self-organizing neural network model for a mechanism of pattern recognition unaffected by shift in positionBiological Cybernetics, 1980