Comprehensive Design Space Exploration for Graph Neural Network Aggregation on GPUs
- 6 February 2025
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Computer Architecture Letters
- Vol. 24 (1) , 45-48
Abstract
Graph neural networks (GNNs) have become the state-of-the-art technology for extracting and predicting data representations on graphs. With increasing demand to accelerate GNN computations, the GPU has become the dominant platform for GNN training and inference. GNNs consist of a compute-bound combination phase and a memory-bound aggregation phase. The memory access patterns of the aggregation phase remain a major performance bottleneck on GPUs, despite recent microarchitectural enhancements. Although GNN characterizations have been conducted to investigate this bottleneck, they did not reveal the impact of architectural modifications. However, a comprehensive understanding of improvements from such modifications is imperative to devise GPU optimizations for the aggregation phase. In this letter, we explore the GPU design space for aggregation by assessing the performance improvement potential of a series of architectural modifications. We find that the low locality of aggregation deteriorates performance with increased thread-level parallelism, and a significant enhancement follows memory access optimizations, which remain effective even with software optimization. Our analysis provides insights for hardware optimizations to significantly improve GNN aggregation on GPUs.Keywords
Funding Information
- Samsung (IO201209-07887-01)
- Institute of Information and Communications Technology Planning and Evaluation
- Korean government (2021-0-00853)
This publication has 12 references indexed in Scilit:
- Architectural Implications of GNN Aggregation Programming AbstractionsIEEE Computer Architecture Letters, 2023
- Analyzing GCN Aggregation on GPUIEEE Access, 2022
- NVIDIA A100 Tensor Core GPU: Performance and InnovationIEEE Micro, 2021
- AWB-GCN: A Graph Convolutional Network Accelerator with Runtime Workload Rebalancing2020
- Accel-Sim: An Extensible Simulation Framework for Validated GPU Modeling2020
- Architectural Implication of Graph Neural NetworksIEEE Computer Architecture Letters, 2020
- Deep Learning on Graphs: A SurveyIEEE Transactions on Knowledge and Data Engineering, 2020
- Characterizing and Understanding GCNs on GPUIEEE Computer Architecture Letters, 2020
- NVBit2019
- DeepGCNs: Can GCNs Go As Deep As CNNs?2019