Eliciting Structural and Semantic Global Knowledge in Unsupervised Graph Contrastive Learning
Open Access
- 26 June 2023
- journal article
- research article
- Published by Association for the Advancement of Artificial Intelligence (AAAI) in Proceedings of the AAAI Conference on Artificial Intelligence
- Vol. 37 (6) , 7378-7386
Abstract
Graph Contrastive Learning (GCL) has recently drawn much research interest for learning generalizable node representations in a self-supervised manner. In general, the contrastive learning process in GCL is performed on top of the representations learned by a graph neural network (GNN) backbone, which transforms and propagates the node contextual information based on its local neighborhoods. However, nodes sharing similar characteristics may not always be geographically close, which poses a great challenge for unsupervised GCL efforts due to their inherent limitations in capturing such global graph knowledge. In this work, we address their inherent limitations by proposing a simple yet effective framework -- Simple Neural Networks with Structural and Semantic Contrastive Learning} (S^3-CL). Notably, by virtue of the proposed structural and semantic contrastive learning algorithms, even a simple neural network can learn expressive node representations that preserve valuable global structural and semantic patterns. Our experiments demonstrate that the node representations learned by S^3-CL) achieve superior performance on different downstream tasks compared with the state-of-the-art unsupervised GCL methods. Implementation and more experimental details are publicly available at https://github.com/kaize0409/S-3-CL.Keywords
All Related Versions
This publication has 11 references indexed in Scilit:
- Toward Graph Minimally-Supervised Learning2022
- Meta Propagation Networks for Graph Few-shot Semi-supervised LearningProceedings of the AAAI Conference on Artificial Intelligence, 2022
- Graph Contrastive Learning with Adaptive Augmentation2021
- Graph Prototypical Networks for Few-shot Learning on Attributed Networks2020
- Next-item Recommendation with Sequential Hypergraphs2020
- Distribution-Induced Bidirectional Generative Adversarial Network for Graph Representation Learning2020
- Momentum Contrast for Unsupervised Visual Representation Learning2020
- Measuring and Relieving the Over-Smoothing Problem for Graph Neural Networks from the Topological ViewProceedings of the AAAI Conference on Artificial Intelligence, 2020
- Symmetric Graph Convolutional Autoencoder for Unsupervised Graph Representation Learning2019
- Deep Anomaly Detection on Attributed Networks2019