Liquid State Machine With Dendritically Enhanced Readout for Low-Power, Neuromorphic VLSI Implementations
- 27 October 2014
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Biomedical Circuits and Systems
- Vol. 8 (5) , 681-695
Abstract
In this paper, we describe a new neuro-inspired, hardware-friendly readout stage for the liquid state machine (LSM), a popular model for reservoir computing. Compared to the parallel perceptron architecture trained by the p-delta algorithm, which is the state of the art in terms of performance of readout stages, our readout architecture and learning algorithm can attain better performance with significantly less synaptic resources making it attractive for VLSI implementation. Inspired by the nonlinear properties of dendrites in biological neurons, our readout stage incorporates neurons having multiple dendrites with a lumped nonlinearity (two compartment model). The number of synaptic connections on each branch is significantly lower than the total number of connections from the liquid neurons and the learning algorithm tries to find the best `combination' of input connections on each branch to reduce the error. Hence, the learning involves network rewiring (NRW) of the readout network similar to structural plasticity observed in its biological counterparts. We show that compared to a single perceptron using analog weights, this architecture for the readout can attain, even by using the same number of binary valued synapses, up to 3.3 times less error for a two-class spike train classification problem and 2.4 times less error for an input rate approximation task. Even with 60 times larger synapses, a group of 60 parallel perceptrons cannot attain the performance of the proposed dendritically enhanced readout. An additional advantage of this method for hardware implementations is that the `choice' of connectivity can be easily implemented exploiting address event representation (AER) protocols commonly used in current neuromorphic systems where the connection matrix is stored in memory. Also, due to the use of binary synapses, our proposed method is more robust against statistical variations.Keywords
All Related Versions
This publication has 36 references indexed in Scilit:
- A Learning-Enabled Neuron Array IC Based Upon Transistor Channel Models of Biological PhenomenaIEEE Transactions on Biomedical Circuits and Systems, 2012
- Continuous Classification of Spatio-temporal Data Streams Using Liquid State Machines2012
- Liquid Computing in a Simplified Model of Cortical Layer IV: Learning to Balance a Ball2012
- Delayed Synapses: An LSM Model for Studying Aspects of Temporal Context in Memory2011
- Spatiotemporal Pattern Recognition via Liquid State Machines2006
- The tempotron: a neuron that learns spike timing–based decisionsNature Neuroscience, 2006
- Computational subunits in thin dendrites of pyramidal cellsNature Neuroscience, 2004
- Impact of Active Dendrites and Structural Plasticity on the Memory Capacity of Neural TissueNeuron, 2001
- New results on recurrent network training: unifying the algorithms and accelerating convergenceIEEE Transactions on Neural Networks, 2000
- Backpropagation through time: what it does and how to do itProceedings of the IEEE, 1990