An FPGA-Efficient CNN Accelerator by Integrating Value-Based Sparsity and Bit-Level Sparsity for Weight Compression
- 12 January 2026
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems
- Vol. PP (02780070) , 1
Abstract
Quantization and sparsity are key techniques for CNN compression and acceleration. Independent exploration of value-based sparsity (VBS) or bit-level sparsity (BLS) in 8b weights yield limited gains. We analyze weight distributions of VGG16 and ResNet50 after 4b quantization and find over 80% zero-values with strong near-structured sparsity. On VGG16, 98.11% of 16-weight groups contain at least 8 zero-values. Based on this analysis, we propose a two-stage sparse compression (TSSC) method that integrates VBS and BLS to efficiently compress weights. In stage 1, we apply post-quantization pruning to enforce structured sparsity with at least 8 zero-values per 16 weights and utilize value-based structured sparsity (VBSS) to compress these 8 zero-values per group. In stage 2, among the remaining weights, we apply column-based BLS (CBBLS) to further compress remaining weights. Finally, a two-stage sparse accelerator, which integrates the TSSC method and corresponding hardware optimization (hardware-efficient activation selection unit and bit-serial multiply-accumulate), is implemented to accelerate the CNN models. Experimental results demonstrate that our proposed TSSC method can reduce the weights of VGG16 and ResNet50 to 0.99b and 0.95b while maintaining an acceptable accuracy drop. When deployed on FPGA, our accelerator achieves an 8.08× weight compression and 7.13× performance improvement compared with 8b non-sparse VGG16 model. Compared with state-of-the-art accelerators, we achieve 1.95×-7.97×, 2.00×-32.8× and 1.13×-6.53× improvement in LUT efficiency, equivalent LUT efficiency and energy efficiency, respectively.Keywords
Funding Information
- Science and Technology Planning Project of Guangzhou (2023B01J0007)
- National Natural Science Foundation of China (62301165)
This publication has 36 references indexed in Scilit:
- An FPGA-based bit-level weight sparsity and mixed-bit accelerator for neural networksJournal of Systems Architecture, 2025
- BitWave: Exploiting Column-Based Bit-Level Sparsity for Deep Learning Acceleration2024
- S2TA: Exploiting Structured Sparsity for Energy-Efficient Mobile CNN Acceleration2022
- A 65-nm Energy-Efficient Interframe Data Reuse Neural Network Accelerator for Video ApplicationsIEEE Journal of Solid-State Circuits, 2021
- SNAP: An Efficient Sparse Neural Acceleration Processor for Unstructured Sparse Deep Neural Network InferenceIEEE Journal of Solid-State Circuits, 2020
- STICKER: An Energy-Efficient Multi-Sparsity Compatible Accelerator for Convolutional Neural Networks in 65-nm CMOSIEEE Journal of Solid-State Circuits, 2019
- Eyeriss v2: A Flexible Accelerator for Emerging Deep Neural Networks on Mobile DevicesIEEE Journal on Emerging and Selected Topics in Circuits and Systems, 2019
- An Efficient Hardware Accelerator for Sparse Convolutional Neural Networks on FPGAs2019
- EIE: Efficient Inference Engine on Compressed Deep Neural Network2016
- Deep Residual Learning for Image Recognition2016