A 65-nm Energy-Efficient Interframe Data Reuse Neural Network Accelerator for Video Applications
- 1 December 2021
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Journal of Solid-State Circuits
- Vol. 57 (8) , 2574-2585
Abstract
An energy-efficient convolutional neural network (CNN) accelerator is proposed for the video application. Previous works exploited the sparsity of differential (Diff) frame activation, but the improvement is limited as many Diff-frame data is small but non-zero. Processing of irregular sparse data also leads to low hardware utilization. To solve these problems, two key innovations are proposed in this article. First, we implement a hybrid-precision inter-frame-reuse architecture which takes advantage of both low bit-width and high sparsity of Diff-frame data. This technology can accelerate 3.2 $\times $ inference speed with no accuracy loss. Second, we design a conv-pattern-aware processing array that achieves the 2.48 $\times $ –14.2 $\times $ PE utilization rate to process sparse data for different convolution kernels. The accelerator chip was implemented in 65-nm CMOS technology. To the best of our knowledge, it is the first silicon-proven CNN accelerator that supports inter-frame data reuse. Attributed to the inter-frame similarity, this video CNN accelerator reaches the minimum energy consumption of 24.7 $\mu \text{J}$ /frame in the MobileNet-slim model, which is 76.3% less than the baseline.
Keywords
Funding Information
- National key research and development program (2018YFA0701500)
- NSFC (61934005)
- Beijing National Research Center For Information Science And Technology
- Beijing Innovation Center for Future Chips
This publication has 35 references indexed in Scilit:
- 14.2 DNPU: An 8.1TOPS/W reconfigurable CNN-RNN processor for general-purpose deep neural networks2017
- Fully-Convolutional Siamese Networks for Object Tracking2016
- Cambricon-X: An accelerator for sparse neural networks2016
- A 0.3–2.6 TOPS/W precision-scalable processor for real-time large-scale ConvNets2016
- Cnvlutin: Ineffectual-Neuron-Free Deep Neural Network Computing2016
- Eyeriss: A Spatial Architecture for Energy-Efficient Dataflow for Convolutional Neural Networks2016
- EIE: Efficient Inference Engine on Compressed Deep Neural Network2016
- Learning Spatiotemporal Features with 3D Convolutional Networks2015
- Going deeper with convolutions2015
- Large-Scale Video Classification with Convolutional Neural Networks2014