When Does Self-Supervision Help Graph Convolutional Networks?
Authors:
Preprint
Abstract
Self-supervision as an emerging technique has been employed to train convolutional neural networks (CNNs) for more transferrable, generalizable, and robust representation learning of images. Its introduction to graph convolutional networks (GCNs) operating on graph data is however rarely explored. In this study, we report the first systematic exploration and assessment of incorporating self-supervision into GCNs. We first elaborate three mechanisms to incorporate self-supervision into GCNs, analyze the limitations of pretraining & finetuning and self-training, and proceed to focus on multi-task learning. Moreover, we propose to investigate three novel self-supervised learning tasks for GCNs with theoretical rationales and numerical comparisons. Lastly, we further integrate multi-task self-supervision into graph adversarial training. Our results show that, with properly designed task forms and incorporation mechanisms, self-supervision benefits GCNs in gaining more generalizability and robustness. Our codes are available at https://github.com/Shen-Lab/SS-GCNs.Keywords
All Related Versions
This publication has 15 references indexed in Scilit:
- Explainable Deep Relational Networks for Predicting Compound–Protein Affinities and ContactsJournal of Chemical Information and Modeling, 2020
- When Does Self-supervision Improve Few-Shot Learning?2020
- Scaling and Benchmarking Self-Supervised Visual Representation Learning2019
- Revisiting Self-Supervised Visual Representation Learning2019
- Bias-Variance Tradeoff of Graph Laplacian RegularizerIEEE Signal Processing Letters, 2017
- Estimating the trace of the matrix inverse by interpolating from the diagonal of an approximate inverseJournal of Computational Physics, 2016
- Seeing the Bigger Picture: How Nodes Can Learn Their Place Within a Complex Ad Hoc Network TopologyIEEE Signal Processing Magazine, 2013
- The emerging field of signal processing on graphs: Extending high-dimensional data analysis to networks and other irregular domainsIEEE Signal Processing Magazine, 2013
- A Tour of Modern Image Filtering: New Insights and Methods, Both Practical and TheoreticalIEEE Signal Processing Magazine, 2012
- Learning Probabilistic Relational Models2001