An Energy-Efficient Low-Latency 3D-CNN Accelerator Leveraging Temporal Locality, Full Zero-Skipping, and Hierarchical Load Balance
- 5 December 2021
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 241-246
Abstract
Three-dimensional convolutional neural network (3D-CNN) has demonstrated outstanding classification performance in video recognition compared to two-dimensional CNN (2D-CNN), since 3D-CNN not only learns the spatial features of each frame, but also learns the temporal features across all frames. However, 3D-CNN suffers from intensive computation and data movement. To solve these issues, an energy-efficient low-latency 3D-CNN accelerator is proposed. Temporal locality and small differential value dropout are used to increase the sparsity of activation. Furthermore, to fully utilize the sparsity of weight and activation, a full zero-skipping convolutional microarchitecture is proposed. A hierarchical load-balancing scheme is also introduced to improve resource utilization. With the proposed techniques, a 3D-CNN accelerator is designed in a 55-nm low-power CMOS technology, bringing in up to 9.89× speedup compared to the baseline implementation. Benchmarked with C3D, the proposed accelerator achieves an energy efficiency of 4.66 TOPS/W at 100 MHz and 1.08 V supply voltage.Keywords
This publication has 14 references indexed in Scilit:
- Convolutional neural networks: an overview and application in radiologyInsights into Imaging, 2018
- UNPU: A 50.6TOPS/W unified deep neural network accelerator with 1b-to-16b fully-variable weight bit-precision2018
- Efficient Processing of Deep Neural Networks: A Tutorial and SurveyProceedings of the IEEE, 2017
- SCNNACM SIGARCH Computer Architecture News, 2017
- CnvlutinACM SIGARCH Computer Architecture News, 2016
- 14.5 Eyeriss: An energy-efficient reconfigurable accelerator for deep convolutional neural networks2016
- Learning Spatiotemporal Features with 3D Convolutional Networks2015
- Large-Scale Video Classification with Convolutional Neural Networks2014
- A 240 G-ops/s Mobile Coprocessor for Deep Neural Networks2014
- 3D Convolutional Neural Networks for Human Action RecognitionIEEE Transactions on Pattern Analysis and Machine Intelligence, 2012