Exploiting Tensor Cores in Sparse Matrix-Multivector Multiplication via Block-Sparsity-Aware Clustering
Authors:
- 27 May 2024
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 1181-1183
Abstract
Sparse Matrix-Multivector (SpMM) multiplication is a key kernel for deep learning models and scientific computing applications. However, achieving high performance for SpMM on GPUs is challenging because of the irregular distribution of non- zero elements and irregular memory accesses in sparse matrices. In this paper, we propose a novel sparse matrix reordering algorithm based on the block sparsity patterns in rows to improve data locality for SpMM. To accelerate our reordering algorithm, we develop a parallel implementation based on GPUs. The high-density tiles in the reordered matrix are used to perform accelerated dense matrix multiplication by leveraging Tensor Cores on GPUs. Experimental results on a large number of sparse matrices demonstrate that our TC-SpMM achieves an average speedup of 3.4x and a peak speedup of 20.77 x over cuSPARSE.Keywords
This publication has 12 references indexed in Scilit:
- Communication Optimization for Distributed Execution of Graph Neural Networks2023
- Blocking Sparse Matrices to Leverage Dense-Specific Multiplication2022
- GE-SpMM: General-Purpose Sparse Matrix-Matrix Multiplication on GPUs for Graph Neural Networks2020
- Sparse GPU Kernels for Deep Learning2020
- In-Datacenter Performance Analysis of a Tensor Processing Unit2017
- Sparse Convolutional Neural Networks2015
- Optimizing Sparse Matrix-Multiple Vectors Multiplication for Nuclear Configuration Interaction Calculations2014
- Model-driven autotuning of sparse matrix-vector multiply on GPUsACM SIGPLAN Notices, 2010
- Parallel sparse matrix-vector and matrix-transpose-vector multiplication using compressed sparse blocks2009
- Finding Exact and Approximate Block Structures for ILU PreconditioningSIAM Journal on Scientific Computing, 2003