Evolver: A Deep Learning Processor With On-Device Quantization–Voltage–Frequency Tuning
- 29 September 2020
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Journal of Solid-State Circuits
- Vol. 56 (2) , 658-673
Abstract
When deploying deep neural networks (DNNs) onto deep learning processors, we usually exploit mixed-precision quantization and voltage-frequency scaling to make tradeoffs among accuracy, latency, and energy. Conventional methods usually determine the quantization-voltage-frequency (QVF) policy before DNNs are deployed onto local devices. However, they are difficult to make optimal customizations for local user scenarios. In this article, we solve the problem by enabling on-device QVF tuning with a new deep learning processor architecture Evolver. Evolver has a QVF tuning mode to deploy DNNs with local customizations before normal execution. In this mode, Evolver uses reinforcement learning to search the optimal QVF policy based on direct hardware feedbacks from the chip itself. After that, Evolver runs the newly quantized DNN inference under the searched voltage and frequency. To improve the performance and energy efficiency of both training and inference, we introduce bidirectional speculation and runtime reconfiguration techniques into the architecture. To the best of our knowledge, Evolver is the first deep learning processor that utilizes on-device QVF tuning to achieve both customized and optimal DNN deployment.Keywords
Funding Information
- National Key R&D Project (2018YFB2202600)
- NSFC (61774094, U19B2041)
- China Major S&T Project (2018ZX01031101-002, 2018ZX01028101-004)
- Beijing S&T Project (Z191100007519016)
This publication has 20 references indexed in Scilit:
- Bit Fusion: Bit-Level Dynamically Composable Architecture for Accelerating Deep Neural Network2018
- UNPU: A 50.6TOPS/W unified deep neural network accelerator with 1b-to-16b fully-variable weight bit-precision2018
- SCNN2017
- 14.5 Envision: A 0.26-to-10TOPS/W subword-parallel dynamic-voltage-accuracy-frequency-scalable Convolutional Neural Network processor in 28nm FDSOI2017
- Stripes: Bit-serial deep neural network computing2016
- Unsupervised Learning of Visual Representations by Solving Jigsaw Puzzles2016
- CnvlutinACM SIGARCH Computer Architecture News, 2016
- Deep Residual Learning for Image Recognition2016
- 14.5 Eyeriss: An energy-efficient reconfigurable accelerator for deep convolutional neural networks2016
- Human-level control through deep reinforcement learningNature, 2015