Long short-term memory and learning-to-learn in networks of spiking neurons
Preprint
Abstract
Recurrent networks of spiking neurons (RSNNs) underlie the astounding computing and learning capabilities of the brain. But computing and learning capabilities of RSNN models have remained poor, at least in comparison with artificial neural networks (ANNs). We address two possible reasons for that. One is that RSNNs in the brain are not randomly connected or designed according to simple rules, and they do not start learning as a tabula rasa network. Rather, RSNNs in the brain were optimized for their tasks through evolution, development, and prior experience. Details of these optimization processes are largely unknown. But their functional contribution can be approximated through powerful optimization methods, such as backpropagation through time (BPTT). A second major mismatch between RSNNs in the brain and models is that the latter only show a small fraction of the dynamics of neurons and synapses in the brain. We include neurons in our RSNN model that reproduce one prominent dynamical process of biological neurons that takes place at the behaviourally relevant time scale of seconds: neuronal adaptation. We denote these networks as LSNNs because of their Long short-term memory. The inclusion of adapting neurons drastically increases the computing and learning capability of RSNNs if they are trained and configured by deep learning (BPTT combined with a rewiring algorithm that optimizes the network architecture). In fact, the computational performance of these RSNNs approaches for the first time that of LSTM networks. In addition RSNNs with adapting neurons can acquire abstract knowledge from prior learning in a Learning-to-Learn (L2L) scheme, and transfer that knowledge in order to learn new but related tasks from very few examples. We demonstrate this for supervised learning and reinforcement learning.Keywords
All Related Versions
This publication has 14 references indexed in Scilit:
- Prefrontal cortex as a meta-reinforcement learning systemNature Neuroscience, 2018
- Generalized leaky integrate-and-fire models classify multiple neuron typesNature Communications, 2018
- Loihi: A Neuromorphic Manycore Processor with On-Chip LearningIEEE Micro, 2018
- Supervised learning in spiking neural networks with FORCE trainingNature Communications, 2017
- Hierarchical process memory: memory as an integral component of information processingTrends in Cognitive Sciences, 2015
- Overview of the SpiNNaker System ArchitectureIEEE Transactions on Computers, 2012
- Spike-Based Reinforcement Learning in Continuous State and Action Space: When Policy Gradient Methods FailPLOS Computational Biology, 2009
- Synaptic Theory of Working MemoryScience, 2008
- Learning long-term dependencies with gradient descent is difficultIEEE Transactions on Neural Networks, 1994
- Developments of a water-maze procedure for studying spatial learning in the ratJournal of Neuroscience Methods, 1984