Representation Learning using Event-based STDP
Preprint
Abstract
Although representation learning methods developed within the framework of traditional neural networks are relatively mature, developing a spiking representation model remains a challenging problem. This paper proposes an event-based method to train a feedforward spiking neural network (SNN) layer for extracting visual features. The method introduces a novel spike-timing-dependent plasticity (STDP) learning rule and a threshold adjustment rule both derived from a vector quantization-like objective function subject to a sparsity constraint. The STDP rule is obtained by the gradient of a vector quantization criterion that is converted to spike-based, spatio-temporally local update rules in a spiking network of leaky, integrate-and-fire (LIF) neurons. Independence and sparsity of the model are achieved by the threshold adjustment rule and by a softmax function implementing inhibition in the representation layer consisting of WTA-thresholded spiking neurons. Together, these mechanisms implement a form of spike-based, competitive learning. Two sets of experiments are performed on the MNIST and natural image datasets. The results demonstrate a sparse spiking visual representation model with low reconstruction loss comparable with state-of-the-art visual coding approaches, yet our rule is local in both time and space, thus biologically plausible and hardware friendly.Keywords
All Related Versions
This publication has 41 references indexed in Scilit:
- Bio-inspired unsupervised learning of visual features leads to robust invariant object recognitionNeurocomputing, 2016
- Evolving spiking neural network—a surveyEvolving Systems, 2013
- A Practical Guide to Training Restricted Boltzmann Machines2012
- Relative spike time coding and STDP-based orientation selectivity in the early visual system in natural continuous and saccadic vision: a computational modelJournal of Computational Neuroscience, 2011
- Evolving spiking neural networks for audiovisual information processingNeural Networks, 2010
- Fast and adaptive network of spiking neurons for multi-view visual pattern recognitionNeurocomputing, 2008
- A network that uses few active neurones to code visual input predicts the diverse shapes of cortical receptive fieldsJournal of Computational Neuroscience, 2006
- Rank Order Coding1998
- Forming sparse representations by local anti-Hebbian learningBiological Cybernetics, 1990
- Simplified neuron model as a principal component analyzerJournal of Mathematical Biology, 1982