FusedMM: A Unified SDDMM-SpMM Kernel for Graph Embedding and Graph Neural Networks
Authors:
- 1 May 2021
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 256-266
Abstract
We develop a fused matrix multiplication kernel that unifies sampled dense-dense matrix multiplication and sparsedense matrix multiplication under a single operation called FusedMM. By using user-defined functions, FusedMM can capture almost all computational patterns needed by popular graph embedding and GNN approaches.FusedMM is an order of magnitude faster than its equivalent kernels in Deep Graph Library. The superior performance of FusedMM comes from the low-level vectorized kernels, a suitable load balancing scheme and an efficient utilization of the memory bandwidth. FusedMM can tune its performance using a code generator and perform equally well on Intel, AMD and ARM processors. FusedMM speeds up an end-to-end graph embedding algorithm by up to 28 x on different processors. The source code is available at https://github.com/HipGraph/FusedMM.Keywords
All Related Versions
Funding Information
- Indiana University
This publication has 13 references indexed in Scilit:
- Adaptive sparse tiling for sparse matrix multiplication2019
- HipMCL: a high-performance parallel implementation of the Markov clustering algorithm for large-scale networksNucleic Acids Research, 2018
- VERSE2018
- GraphMatProceedings of the VLDB Endowment, 2015
- Parallel Triangle Counting and Enumeration Using Matrix Algebra2015
- ForceAtlas2, a Continuous Graph Layout Algorithm for Handy Network Visualization Designed for the Gephi SoftwarePLOS ONE, 2014
- Ligra2013
- Graph Algorithms in the Language of Linear Algebra2011
- Pregel2010
- RooflineCommunications of the ACM, 2009