SHARP: An Adaptable, Energy-Efficient Accelerator for Recurrent Neural Networks
Open Access
- 24 January 2023
- journal article
- research article
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Embedded Computing Systems
- Vol. 22 (2) , 1-23
Abstract
The effectiveness of Recurrent Neural Networks (RNNs) for tasks such as Automatic Speech Recognition has fostered interest in RNN inference acceleration. Due to the recurrent nature and data dependencies of RNN computations, prior work has designed customized architectures specifically tailored to the computation pattern of RNN, getting high computation efficiency for certain chosen model sizes. However, given that the dimensionality of RNNs varies a lot for different tasks, it is crucial to generalize this efficiency to diverse configurations. In this work, we identify adaptiveness as a key feature that is missing from today’s RNN accelerators. In particular, we first show the problem of low resource utilization and low adaptiveness for the state-of-the-art RNN implementations on GPU, FPGA, and ASIC architectures. To solve these issues, we propose an intelligent tiled-based dispatching mechanism for increasing the adaptiveness of RNN computation, in order to efficiently handle the data dependencies. To do so, we propose Sharp as a hardware accelerator, which pipelines RNN computation using an effective scheduling scheme to hide most of the dependent serialization. Furthermore, Sharp employs dynamic reconfigurable architecture to adapt to the model’s characteristics. Sharp achieves 2×, 2.8×, and 82× speedups on average, considering different RNN models and resource budgets, compared to the state-of-the-art ASIC, FPGA, and GPU implementations, respectively. Furthermore, we provide significant energy reduction with respect to the previous solutions, due to the low power dissipation of Sharp (321 GFLOPS/Watt).Keywords
All Related Versions
Funding Information
- CoCoUnit ERC Advanced
- EU’s Horizon 2020 (833057)
- Spanish State Research Agency (PID2020-113172RB-I00)
- ICREA Academia
This publication has 14 references indexed in Scilit:
- GRNN2019
- E-PUR2018
- A Configurable Cloud-Scale DNN Processor for Real-Time AI2018
- An Ultra Low-Power Hardware Accelerator for Acoustic Scoring in Speech Recognition2017
- In-Datacenter Performance Analysis of a Tensor Processing Unit2017
- FPGA-based accelerator for long short-term memory recurrent neural networks2017
- Beyond short snippets: Deep networks for video classification2015
- FPGA Acceleration of Recurrent Neural Network Based Language Model2015
- Long Short-Term MemoryNeural Computation, 1997
- Algorithm AS 136: A K-Means Clustering AlgorithmJournal of the Royal Statistical Society Series C: Applied Statistics, 1979