GE-SpMM: General-Purpose Sparse Matrix-Matrix Multiplication on GPUs for Graph Neural Networks
Authors:
Top Cited Papers
- 1 November 2020
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 1-12
Abstract
The acceleration of Graph Neural Networks (GNNs) requires efficient and framework-compatible Sparse-Dense Matrix-Matrix Multiplication (SpMM). From the compatibility perspective, the sophisticated sparse matrix representations in state-of-the-art SpMM designs cause heavy preprocessing overhead for the framework. From the efficiency perspective, optimizations for SpMV (Sparse Matrix-Vector) do not apply well to SpMM, leading to redundant and uncoalesced global memory access. We propose GE-SpMM1, which takes the CSR format consistent with GNN frameworks to enable integration without the format transformation overhead. We use Coalesced Row Caching to ensure coalesced access to both sparse and dense data in the global memory. We use Coarse-grained Warp Merging to reduce redundant data loading among GPU warps. Experiments on a real-world graph dataset demonstrate up to 1.41× speedup over Nvidia cuSPARSE [1] and up to 1.81× over GraphBLAST [2]. We embed GE-SpMM in GNN frameworks and get up to 3.67× speedup on popular GNN models like GCN [3] and GraphSAGE [4].Keywords
All Related Versions
Funding Information
- Research and Development
- National Natural Science Foundation of China
- China Postdoctoral Science Foundation
- Beijing Innovation Center for Future Chip
This publication has 14 references indexed in Scilit:
- Graph Convolutional Neural Networks for Web-Scale Recommender Systems2018
- Efficient sparse-matrix multi-vector product on GPUs2018
- Modeling Relational Data with Graph Convolutional Networks2018
- Gunrock2016
- FastSpMM: An Efficient Library for Sparse Matrix Matrix Product on GPUsThe Computer Journal, 2013
- Ligra2013
- A unified optimizing compiler framework for different GPGPU architecturesACM Transactions on Architecture and Code Optimization, 2012
- The university of Florida sparse matrix collectionACM Transactions on Mathematical Software, 2011
- Implementing sparse matrix-vector multiplication on throughput-oriented processors2009
- Benchmarking GPUs to tune dense linear algebra2008