RELL-STC: An Efficient SpMM Extension of Sparse Tensor Core with Re-structured ELLPACK and Gustavson Dataflow
Open Access
- 20 July 2026
- journal article
- Published by Association for Computing Machinery (ACM) in ACM SIGMETRICS Performance Evaluation Review
- Vol. 54 (1) , 129-131
Abstract
Sparsity is crucial for deep neural networks, reducing storage and bandwidth while improving computational efficiency. With the rapid development of deep neural networks, sparse computation accelerators have become a key research focus. NVIDIA introduced 2:4 fine-grained structured sparsity in the Ampere architecture, doubling Tensor Core peak throughput by retaining two out of four elements, but this may compromise model accuracy. In contrast, unstructured sparsity achieves maximum sparsity without accuracy loss. However, the irregular distribution of non-zero elements (NNZs) in unstructured sparse matrices causes complex memory access and load imbalance, preventing direct use of GPU Tensor Cores. Thus, computation relies on CUDA cores via sparse libraries like cuSPARSE, underutilizing Tensor Cores' capabilities. To address this, we extend the Tensor Core architecture based on Gustavson's dataflow and propose a re-structured row offline compression format for inference. This format integrates with the extended architecture to enable efficient unstructured sparse-dense matrix multiplication. Experiments show negligible area and power overheads, with an average speedup of 3.54x over cuSPARSE and 1.2x over state-of-the-art Tensor Core extensions.Keywords
This publication has 7 references indexed in Scilit:
- RM-STC: Row-Merge Dataflow Inspired GPU Sparse Tensor Core for Energy-Efficient Sparse Acceleration2023
- Dual-side Sparse Tensor Core2021
- Accel-Sim: An Extensible Simulation Framework for Validated GPU Modeling2020
- In-Datacenter Performance Analysis of a Tensor Processing Unit2017
- Analyzing CUDA workloads using a detailed GPU simulator2009
- FreePDK: An Open-Source Variation-Aware Design Kit2007
- Two Fast Algorithms for Sparse Matrices: Multiplication and Permuted TranspositionACM Transactions on Mathematical Software, 1978