An Efficient Bit-Sparse DNN Accelerator Exploiting Adaptive Bit-Serial Computations

Bit sparsity, an intrinsic attribute of binary representation, has been widely utilized in deep neural network (DNN) inference acceleration. Despite the advantages in performance and energy efficiency demonstrated by existing bit-serial-based bit-sparse accelerators, they still face two notable limitations: 1) at the low-level bit-serial multiplier level, existing methods either statically select weight or activation as the serialized object during the design phase, or simply serialize both without considering the distribution of nonzero bits in different operands, thereby failing to achieve optimal performance and 2) at the high-level dataflow level, existing approaches do not eliminate zero values in data movement and computation, leading to considerable energy and latency overhead, as well as suboptimal PE utilization. In this work, we propose AdaS-Pro accelerator for fast and energy-efficient DNN inference. At the multiplier level, AdaS-Pro employs an adaptive bit-serial computation scheme, which dynamically serializes the input operand with fewer nonzero bits at runtime, thereby minimizing compute cycles. To further enhance performance, AdaS-Pro introduces an improved Booth encoding method to reduce the number of nonzero bits in each operand. At the dataflow level, AdaS-Pro employs a compressed format to eliminate zero values and proposes a bi-directional inner-join unit coupled with a ring-shaped scheduler to achieve efficient nonzero workload extraction and balancing. Experimental results show that AdaS-Pro outperforms existing state-of-the-art bit-sparse accelerators, such as BitLet, BitX, and Laconic, with performance improvements of $4.03\times $ , $6.78\times $ , and $1.43\times $ , respectively.