S2TA: Exploiting Structured Sparsity for Energy-Efficient Mobile CNN Acceleration
Authors:
Top Cited Papers
- 1 April 2022
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 573-586
Abstract
Exploiting sparsity is a key technique in accelerating quantized convolutional neural network (CNN) inference on mobile devices. Prior sparse CNN accelerators largely exploit unstructured sparsity and achieve significant speedups. Due to the unbounded, largely unpredictable sparsity patterns, however, exploiting unstructured sparsity requires complicated hardware design with significant energy and area overhead, which is particularly detrimental to mobile/IoT inference scenarios where energy and area efficiency are crucial.We propose to exploit structured sparsity, more specifically, Density Bound Block (DBB) sparsity for both weights and activations. DBB block tensors bound the maximum number of non-zeros per block. DBB thus exposes statically predictable sparsity patterns that enable lean sparsity-exploiting hardware and efficient memory access. We propose new hardware primitives to implement DBB sparsity for (static) weights and (dynamic) activations, respectively, with very low overheads.Building on top of the primitives, we describe S2TA, a systolic array-based CNN accelerator that exploits joint weight and activation DBB sparsity and new dimensions of data reuse unavailable on the traditional systolic array. S2TA in 16nm achieves more than 2× speedup and energy reduction compared to a strong baseline of a systolic array with zero-value clock gating, over five popular CNN benchmarks. Compared to two recent non-systolic sparse accelerators, Eyeriss v2 (65nm) and SparTen (45nm), S2TA in 65nm uses about 2.2× per and 3.1× less energy inference, respectively.Keywords
All Related Versions
This publication has 31 references indexed in Scilit:
- OuterSPACE: An Outer Product Based Sparse Matrix Multiplication Accelerator2018
- Applications of Deep Neural Networks for Ultra Low Power IoT2017
- Bit-pragmatic deep neural network computing2017
- SCNN2017
- In-Datacenter Performance Analysis of a Tensor Processing Unit2017
- CnvlutinACM SIGARCH Computer Architecture News, 2016
- MinervaACM SIGARCH Computer Architecture News, 2016
- EyerissACM SIGARCH Computer Architecture News, 2016
- EIEACM SIGARCH Computer Architecture News, 2016
- Why systolic architectures?Computer, 1982