SHARP: An Adaptable, Energy-Efficient Accelerator for Recurrent Neural Network
Preprint
Abstract
The effectiveness of Recurrent Neural Networks (RNNs) for tasks such as Automatic Speech Recognition has fostered interest in RNN inference acceleration. Due to the recurrent nature and data dependencies of RNN computations, prior work has designed customized architectures specifically tailored to the computation pattern of RNN, getting high computation efficiency for certain chosen model sizes. However, given that the dimensionality of RNNs varies a lot for different tasks, it is crucial to generalize this efficiency to diverse configurations. In this work, we identify adaptiveness as a key feature that is missing from today's RNN accelerators. In particular, we first show the problem of low resource-utilization and low adaptiveness for the state-of-the-art RNN implementations on GPU, FPGA and ASIC architectures. To solve these issues, we propose an intelligent tiled-based dispatching mechanism for increasing the adaptiveness of RNN computation, in order to efficiently handle the data dependencies. To do so, we propose Sharp as a hardware accelerator, which pipelines RNN computation using an effective scheduling scheme to hide most of the dependent serialization. Furthermore, Sharp employs dynamic reconfigurable architecture to adapt to the model's characteristics. Sharp achieves 2x, 2.8x, and 82x speedups on average, considering different RNN models and resource budgets, compared to the state-of-the-art ASIC, FPGA, and GPU implementations, respectively. Furthermore, we provide significant energy-reduction with respect to the previous solutions, due to the low power dissipation of Sharp (321 GFLOPS/Watt).Keywords
All Related Versions
This publication has 11 references indexed in Scilit:
- SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognition2019
- Towards Non-Saturating Recurrent Units for Modelling Long-Term DependenciesProceedings of the AAAI Conference on Artificial Intelligence, 2019
- Gaussian Process Lstm Recurrent Neural Network Language Models for Speech Recognition2019
- Learning to Ask Unanswerable Questions for Machine Reading Comprehension2019
- A Configurable Cloud-Scale DNN Processor for Real-Time AI2018
- Residual LSTM: Design of a Deep Recurrent Architecture for Distant Speech Recognition2017
- Long-Term Recurrent Convolutional Networks for Visual Recognition and DescriptionIEEE Transactions on Pattern Analysis and Machine Intelligence, 2016
- EESEN: End-to-end speech recognition using deep RNN models and WFST-based decoding2015
- Beyond short snippets: Deep networks for video classification2015
- Show and tell: A neural image caption generator2015