Beyond Localized Graph Neural Networks: An Attributed Motif Regularization Framework
Authors:
- 1 November 2020
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 472-481
Abstract
We present InfoMotif, a new semi-supervised, motif-regularized, learning framework over graphs. We overcome two key limitations of message passing in popular graph neural networks (GNNs): localization (a k-layer GNN cannot utilize features outside the k-hop neighborhood of the labeled training nodes) and over-smoothed (structurally indistinguishable) representations. We propose the concept of attributed structural roles of nodes based on their occurrence in different network motifs, independent of network proximity. Two nodes share attributed structural roles if they participate in topologically similar motif instances over co-varying sets of attributes. Further, InfoMotif achieves architecture independence by regularizing the node representations of arbitrary GNNs via mutual information maximization. Our training curriculum dynamically prioritizes multiple motifs in the learning process without relying on distributional assumptions in the underlying graph or the learning task. We integrate three state-of-the-art GNNs in our framework, to show significant gains (3–10% accuracy) across six diverse, real-world datasets. We see stronger gains for nodes with sparse training labels and diverse attributes in local neighborhood structures.Keywords
All Related Versions
This publication has 22 references indexed in Scilit:
- Learning Structural Node Embeddings via Diffusion Wavelets2018
- Multiscale mixing patterns in networksProceedings of the National Academy of Sciences, 2018
- Dual Graph Convolutional Networks for Graph-Based Semi-Supervised Classification2018
- struc2vec2017
- Motifs in Temporal Networks2017
- node2vec2016
- RolX2012
- Collective Classification in Network DataAI Magazine, 2008
- Network Motifs: Simple Building Blocks of Complex NetworksScience, 2002
- Statistical mechanics of complex networksReviews of Modern Physics, 2002