Generating Adaptive Behaviour within a Memory-Prediction Framework
Open Access
- 17 January 2012
- journal article
- research article
- Published by Public Library of Science (PLoS) in PLOS ONE
- Vol. 7 (1) , e29264
Abstract
The Memory-Prediction Framework (MPF) and its Hierarchical-Temporal Memory implementation (HTM) have been widely applied to unsupervised learning problems, for both classification and prediction. To date, there has been no attempt to incorporate MPF/HTM in reinforcement learning or other adaptive systems; that is, to use knowledge embodied within the hierarchy to control a system, or to generate behaviour for an agent. This problem is interesting because the human neocortex is believed to play a vital role in the generation of behaviour, and the MPF is a model of the human neocortex. We propose some simple and biologically-plausible enhancements to the Memory-Prediction Framework. These cause it to explore and interact with an external world, while trying to maximize a continuous, time-varying reward function. All behaviour is generated and controlled within the MPF hierarchy. The hierarchy develops from a random initial configuration by interaction with the world and reinforcement learning only. Among other demonstrations, we show that a 2-node hierarchy can learn to successfully play “rocks, paper, scissors” against a predictable opponent.Keywords
This publication has 27 references indexed in Scilit:
- What do the basal ganglia do? A modeling perspectiveBiological Cybernetics, 2010
- Conditional routing of information to the cortex: A model of the basal ganglia’s role in cognitive coordination.Psychological Review, 2010
- Towards a Mathematical Theory of Cortical Micro-circuitsPLOS Computational Biology, 2009
- Sequence memory for prediction, inference and behaviourPhilosophical Transactions of the Royal Society B: Biological Sciences, 2009
- The cortico-basal ganglia integrative network: The role of the thalamusBrain Research Bulletin, 2009
- Robust Character Recognition Using a Hierarchical Bayesian NetworkLecture Notes in Computer Science, 2006
- Hierarchical Bayesian inference in the visual cortexJournal of the Optical Society of America A, 2003
- Between MDPs and semi-MDPs: A framework for temporal abstraction in reinforcement learningArtificial Intelligence, 1999
- Reinforcement Learning: An IntroductionIEEE Transactions on Neural Networks, 1998
- A network that learns to recognize three-dimensional objectsNature, 1990