An Efficient and Flexible Accelerator Design for Sparse Convolutional Neural Networks
- 20 May 2021
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Circuits and Systems I: Regular Papers
- Vol. 68 (7) , 2936-2949
Abstract
Designing hardware accelerators for convolutional neural networks (CNNs) has recently attracted tremendous attention. Plenty of existing accelerators are built for dense CNNs or structured sparse CNNs. By contrast, unstructured sparse CNNs can achieve higher compression ratio with equivalent accuracy. However, their corresponding hardware implementations generally suffer from load imbalance and conflict access to on-chip buffers, which results in under utilization of processing elements (PEs). To tackle these issues, we propose a hardware/power-efficient and highly flexible architecture to support both unstructured and structured sparse CNNs with various configurations. Firstly, we propose an efficient weight reordering algorithm to preprocess compressed weights and balance the workload of PEs. Secondly, an adaptive on-chip dataflow, namely hybrid parallel (HP) dataflow, is introduced to promote weight reuse. Thirdly, the partial fusion scheme, which was first introduced in one of our prior works, is incorporated as the off-chip dataflow. Benefited from dataflow optimizations, the repetitive data exchanges between on-chip buffers and external memories are significantly reduced. We implement the design on the Intel Arria10 SX660 platform and evaluate with MobileNet-v2, ResNet-50, and ResNet-18 on ImageNet dataset. Compared to existing sparse accelerators on FPGAs, the proposed accelerator can achieve 1.35 ~ 1.81× improvement in power efficiency with the same sparsity. Compared to prior dense accelerators, this accelerator can achieve an improvement of 1.92 ~ 5.84× in DSP efficiency.Keywords
Funding Information
- National Natural Science Foundation of China (61774082)
- Fundamental Research Funds for the Central Universities (021014380148, 021014380137)
- Key Research Plan of Jiangsu Province of China (BE2019003-4)
This publication has 34 references indexed in Scilit:
- Eyeriss: An Energy-Efficient Reconfigurable Accelerator for Deep Convolutional Neural NetworksIEEE Journal of Solid-State Circuits, 2016
- Fused-layer CNN accelerators2016
- EyerissACM SIGARCH Computer Architecture News, 2016
- EIEACM SIGARCH Computer Architecture News, 2016
- Deep Residual Learning for Image Recognition2016
- Fast Algorithms for Convolutional Neural Networks2016
- You Only Look Once: Unified, Real-Time Object Detection2016
- Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification2015
- Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation2014
- RooflineCommunications of the ACM, 2009