Liquid State Machine with Dendritically Enhanced Readout for Low-power, Neuromorphic VLSI Implementations
Authors:
- Subhrajit RoySRSubhrajit Roy
- Amitava BanerjeeABAmitava Banerjee
- Arindam BasuABArindam Basu
Preprint
Abstract
In this paper, we describe a new neuro-inspired, hardware-friendly readout stage for the liquid state machine (LSM), a popular model for reservoir computing. Compared to the parallel perceptron architecture trained by the p-delta algorithm, which is the state of the art in terms of performance of readout stages, our readout architecture and learning algorithm can attain better performance with significantly less synaptic resources making it attractive for VLSI implementation. Inspired by the nonlinear properties of dendrites in biological neurons, our readout stage incorporates neurons having multiple dendrites with a lumped nonlinearity. The number of synaptic connections on each branch is significantly lower than the total number of connections from the liquid neurons and the learning algorithm tries to find the best 'combination' of input connections on each branch to reduce the error. Hence, the learning involves network rewiring (NRW) of the readout network similar to structural plasticity observed in its biological counterparts. We show that compared to a single perceptron using analog weights, this architecture for the readout can attain, even by using the same number of binary valued synapses, up to 3.3 times less error for a two-class spike train classification problem and 2.4 times less error for an input rate approximation task. Even with 60 times larger synapses, a group of 60 parallel perceptrons cannot attain the performance of the proposed dendritically enhanced readout. An additional advantage of this method for hardware implementations is that the 'choice' of connectivity can be easily implemented exploiting address event representation (AER) protocols commonly used in current neuromorphic systems where the connection matrix is stored in memory. Also, due to the use of binary synapses, our proposed method is more robust against statistical variations.Keywords
All Related Versions
This publication has 22 references indexed in Scilit:
- Use of the separation property to derive Liquid State Machines with enhanced classification performanceNeurocomputing, 2013
- Effects of synaptic connectivity on liquid state machine performanceNeural Networks, 2013
- Electrical parameters influence on the dynamics of the Hodgkin–Huxley liquid state machineNeurocomputing, 2012
- A learning rule for very simple universal approximators consisting of a single layer of perceptronsNeural Networks, 2008
- Isolated word recognition with the Liquid State Machine: a case studyInformation Processing Letters, 2005
- Computational subunits in thin dendrites of pyramidal cellsNature Neuroscience, 2004
- Computer Models and Analysis Tools for Neural Microcircuits2003
- Impact of Active Dendrites and Structural Plasticity on the Memory Capacity of Neural TissueNeuron, 2001
- Fast training of recurrent networks based on the EM algorithmIEEE Transactions on Neural Networks, 1998
- Neurocontrol of nonlinear dynamical systems with Kalman filter trained recurrent networksIEEE Transactions on Neural Networks, 1994