Generalised Analog LSTMs Recurrent Modules for Neural Computing
Authors:
Open Access
- 28 September 2021
- journal article
- research article
- Published by Frontiers Media SA in Frontiers in Computational Neuroscience
- Vol. 15
Abstract
The human brain can be considered as a complex dynamic and recurrent neural network. There are several models for neural networks of the human brain, that cover sensory to cortical information processing. Large majority models include feedback mechanisms that are hard to formalise to realistic applications. Recurrent neural networks and Long short-term memory (LSTM) inspire from the neuronal feedback networks. Long short-term memory (LSTM) prevent vanishing and exploding gradients problems faced by simple recurrent neural networks and has the ability to process order-dependent data. Such recurrent neural units can be replicated in hardware and interfaced with analog sensors for efficient and miniaturised implementation of intelligent processing. Implementation of analog memristive LSTM hardware is an open research problem and can offer the advantages of continuous domain analog computing with relatively low on-chip area compared with a digital-only implementation. Designed for solving time-series prediction problems, overall architectures and circuits were tested with TSMC 0.18 μm CMOS technology and hafnium-oxide (HfO2) based memristor crossbars. Extensive circuit based SPICE simulations with over 3,500 (inference only) and 300 system-level simulations (training and inference) were performed for benchmarking the system performance of the proposed implementations. The analysis includes Monte Carlo simulations for the variability of memristors' conductance, and crossbar parasitic, where non-idealities of hybrid CMOS-memristor circuits are taken into the account.Keywords
This publication has 16 references indexed in Scilit:
- Analogue signal and image processing with large memristor crossbarsNature Electronics, 2017
- On-chip training of memristor crossbar based multi-layer neural networksMicroelectronics Journal, 2017
- ESE2017
- FPGA-based accelerator for long short-term memory recurrent neural networks2017
- An FPGA implementation of a long short-term memory neural network2016
- Investigating the switching dynamics and multilevel capability of bipolar metal oxide resistive switching memoryApplied Physics Letters, 2011
- Indirect compensation techniques for three-stage fully-differential op-amps2010
- Prediction of top-oil temperature for transformers using neural networksIEEE Transactions on Power Delivery, 2000
- A Symmetric Complementary Structure for RF CMOS Analog Squarer and Four-Quadrant Analog MultiplierAnalog Integrated Circuits and Signal Processing, 2000
- Long Short-Term MemoryNeural Computation, 1997