Similarity-Based LSTM Architecture for Energy-Efficient Edge-Level Speech Recognition
Authors:
- 1 July 2019
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 1-6
Abstract
Targeting the resource-limited edge devices, we present a novel processing architecture of long short-term memory (LSTM) networks for low-power speech recognition. The proposed scheme newly defines the similarity score between two inputs of adjacent LSTM cells, and then the processing mode of the current LSTM cell is dynamically determined to reduce the energy while providing the accurate recognition. If the similarity is high, more precisely, the current cell is disabled and the outputs are directly copied from the prior vectors, totally eliminating complex LSTM operations. To maximize the skipping ratio without degrading the accuracy, for the first time, we analyze the effects of skipping the consecutive cells and set the upper limit of the number of consecutive skips. When two adjacent inputs are weakly similar, in addition, we modify the concept of the previous delta-computing, which approximately activate the LSTM cell with low computational resolution, further reducing the energy consumption. Compared to the previous state-of-the-art solutions, as a result, the proposed LSTM architecture remarkably saves the energy consumed for the accurate speech recognition, which is suitable to the resource-limited embedded edges.Keywords
This publication has 13 references indexed in Scilit:
- Deep Convolutional Neural Network Architecture With Reconfigurable Computation PatternsIEEE Transactions on Very Large Scale Integration (VLSI) Systems, 2017
- ESE2017
- Accelerating recurrent neural networks in analytics servers: Comparison of FPGA, CPU, GPU, and ASIC2016
- Dynamic Approximation with Feedback Control for Energy-Efficient Recurrent Neural Network Hardware2016
- On the compression of recurrent neural networks with an application to LVCSR acoustic modeling for embedded speech recognition2016
- Librispeech: An ASR corpus based on public domain audio books2015
- Hybrid speech recognition with Deep Bidirectional LSTM2013
- Speech recognition with deep recurrent neural networks2013
- Long Short-Term MemoryNeural Computation, 1997
- The Use of Recurrent Neural Networks in Continuous Speech Recognition1996