Enhancing Graph Neural Networks via Memorized Global Information
Open Access
- 8 October 2024
- journal article
- research article
- Published by Association for Computing Machinery (ACM) in ACM Transactions on the Web
- Vol. 18 (4) , 1-34
Abstract
Graph neural networks (GNNs) have gained significant attention for their impressive results on different graph-based tasks. The essential mechanism of GNNs is the message-passing framework, whereby node representations are aggregated from local neighborhoods. Recently, Transformer-based GNNs have been introduced to learn the long-range dependencies, enhancing performance. However, their quadratic computational complexity, due to the attention computation, has constrained their applicability on large-scale graphs. To address this issue, we propose MGIGNN ( M emorized G lobal I nformation G raph N eural N etwork), an innovative approach that leverages memorized global information to enhance existing GNNs in both transductive and inductive scenarios. Specifically, MGIGNN captures long-range dependencies by identifying and incorporating global similar nodes, which are defined as nodes exhibiting similar features, structural patterns and label information within a graph. To alleviate the computational overhead associated with computing embeddings for all nodes, we introduce an external memory module to facilitate the retrieval of embeddings and optimize performance on large graphs. To enhance the memory-efficiency, MGIGNN selectively retrieves global similar nodes from a small set of candidate nodes. These candidate nodes are selected from the training nodes based on a sparse node selection distribution with a Dirichlet prior. This selecting approach not only reduces the memory size required but also ensures efficient utilization of computational resources. Through comprehensive experiments conducted on ten widely-used and real-world datasets, including seven homogeneous datasets and three heterogeneous datasets, we demonstrate that our MGIGNN can generally improve the performance of existing GNNs on node classification tasks under both inductive and transductive settings.Keywords
This publication has 37 references indexed in Scilit:
- Graph neural network for traffic forecasting: A surveyExpert Systems with Applications, 2022
- Topology-Aware Graph Pooling NetworksIEEE Transactions on Pattern Analysis and Machine Intelligence, 2021
- Context-Dependent Propagating-Based Video Recommendation in Multimodal Heterogeneous Information NetworksIEEE Transactions on Multimedia, 2020
- MAGNN: Metapath Aggregated Graph Neural Network for Heterogeneous Graph Embedding2020
- Heterogeneous Graph Transformer2020
- Cluster-GCN2019
- Heterogeneous Graph Attention Network2019
- Predicting multicellular function through multi-layer tissue networksBioinformatics, 2017
- ArnetMiner2008
- The Benefits of Facebook “Friends:” Social Capital and College Students’ Use of Online Social Network SitesJournal of Computer-Mediated Communication, 2007