FusedMM: A Unified SDDMM-SpMM Kernel for Graph Embedding and Graph Neural Networks
Authors:
- Khaledur RahmanKRKhaledur Rahman
- Majedul Haque SujonMSMajedul Haque Sujon
- Ariful AzadAAAriful Azad
Preprint
Abstract
We develop a fused matrix multiplication kernel that unifies sampled dense-dense matrix multiplication and sparse-dense matrix multiplication under a single operation called FusedMM. By using user-defined functions, FusedMM can capture almost all computational patterns needed by popular graph embedding and GNN approaches. FusedMM is an order of magnitude faster than its equivalent kernels in Deep Graph Library. The superior performance of FusedMM comes from the low-level vectorized kernels, a suitable load balancing scheme and an efficient utilization of the memory bandwidth. FusedMM can tune its performance using a code generator and perform equally well on Intel, AMD and ARM processors. FusedMM speeds up an end-to-end graph embedding algorithm by up to 28x on different processors.Keywords
All Related Versions
This publication has 8 references indexed in Scilit:
- GE-SpMM: General-Purpose Sparse Matrix-Matrix Multiplication on GPUs for Graph Neural Networks2020
- FeatGraph: A Flexible and Efficient Backend for Graph Neural Network Systems2020
- Sampled Dense Matrix Multiplication for High-Performance Machine Learning2018
- HipMCL: a high-performance parallel implementation of the Markov clustering algorithm for large-scale networksNucleic Acids Research, 2018
- Scalable SIMD-Efficient Graph Processing on GPUs2015
- Parallel Triangle Counting and Enumeration Using Matrix Algebra2015
- ForceAtlas2, a Continuous Graph Layout Algorithm for Handy Network Visualization Designed for the Gephi SoftwarePLOS ONE, 2014
- The Combinatorial BLAS: design, implementation, and applicationsThe International Journal of High Performance Computing Applications, 2011