DeeperGCN: Training Deeper GCNs with Generalized Aggregation Functions
- 21 August 2023
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Pattern Analysis and Machine Intelligence
- Vol. PP (01628828) , 1-12
Abstract
Graph Neural Networks (GNNs) have been drawing significant attention to representation learning on graphs. Recent works developed frameworks to train very deep GNNs and showed impressive results in tasks like point cloud learning and protein interaction prediction. In this work, we study the performance of such deep models in large-scale graphs. In particular, we look at the effect of adequately choosing an aggregation function on deep models. We find that GNNs are very sensitive to the choice of aggregation functions (e.g. mean, max, and sum) when applied to different datasets. We systematically study and propose to alleviate this issue by introducing a novel class of aggregation functions named Generalized Aggregation Functions. The proposed functions extend beyond commonly used aggregation functions to a wide range of new permutation-invariant functions. Generalized Aggregation Functions are fully differentiable, where their parameters can be learned in an end-to-end fashion to yield a suitable aggregation function for each task. We show that equipped with the proposed aggregation functions, deep residual GNNs outperform state-of-the-art in several benchmarks from Open Graph Benchmark (OGB) across tasks and domains. The code and models for reproducing our experiments are available at https://github.com/lightaime/deep_gcns_torch/tree/master/examples/ogb.Keywords
This publication has 20 references indexed in Scilit:
- Improving CNN linear layers with power mean non-linearityPattern Recognition, 2019
- CayleyNets: Graph Convolutional Neural Networks With Complex Rational Spectral FiltersIEEE Transactions on Signal Processing, 2018
- Large-Scale Learnable Graph Convolutional Networks2018
- Densely Connected Convolutional Networks2017
- Geometric Deep Learning on Graphs and Manifolds Using Mixture Model CNNs2017
- Identity Mappings in Deep Residual Networks2016
- node2vec2016
- Deep Residual Learning for Image Recognition2016
- DeepWalk2014
- The Graph Neural Network ModelIEEE Transactions on Neural Networks, 2008