IGCN: A Provably Informative GCN Embedding for Semi-Supervised Learning With Extremely Limited Labels
- 23 May 2024
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Pattern Analysis and Machine Intelligence
- Vol. 46 (12) , 8396-8409
Abstract
Graph Neural Networks (GNNs) have gained much more attention in the representation learning for the graph-structured data. However, the labels are always limited in the graph, which easily leads to the overfitting problem and causes the poor performance. To solve this problem, we propose a new framework called IGCN, short for Informative Graph Convolutional Network, where the objective of IGCN is designed to obtain the informative embeddings via discarding the task-irrelevant information of the graph data based on the mutual information. As the mutual information for irregular data is intractable to compute, our framework is optimized via a surrogate objective, where two terms are derived to approximate the original objective. For the former term, it demonstrates that the mutual information between the learned embeddings and the ground truth should be high, where we utilize the semi-supervised classification loss and the prototype based supervised contrastive learning loss for optimizing it. For the latter term, it requires that the mutual information between the learned node embeddings and the initial embeddings should be high and we propose to minimize the reconstruction loss between them to achieve the goal of maximizing the latter term from the feature level and the layer level, which contains the graph encoder-decoder module and a novel architecture GCN $_{Info}$ . Moreover, we provably show that the designed GCN $_{Info}$ can better alleviate the information loss and preserve as much useful information of the initial embeddings as possible. Experimental results show that the IGCN outperforms the state-of-the-art methods on 7 popular datasets.
Keywords
Funding Information
- National Natural Science Foundation of China (61991411)
- National Science and Technology Major (2021ZD0112002)
- National Natural Science Foundation of China (62076148)
- Shandong Excellent Young Scientists Fund Program (2022HWYQ-042)
- Young Taishan Scholars Program of Shandong Province (tsqn201909029)
- Self-Developed Innovation Team of Jinan City (2021GXRC038)
This publication has 24 references indexed in Scilit:
- Informative pseudo-labeling for graph neural networks with few labelsData Mining and Knowledge Discovery, 2022
- Node Representation Learning in Graph via Node-to-Neighbourhood Mutual Information Maximization2022
- Self-SAGCN: Self-Supervised Semantic Alignment for Graph Convolution Network2021
- Shoestring: Graph-Based Semi-Supervised Classification With Severely Limited Labeled Data2020
- Semi-Supervised Learning With Graph Learning-Convolutional Networks2019
- Deep Spectral Clustering Using Dual Autoencoder Network2019
- Graph Convolutional Networks: Algorithms, Applications and Open Challenges2018
- Learning and generalization with the information bottleneckTheoretical Computer Science, 2010
- Regularization and Semi-supervised Learning on Large Graphs2004
- Automating the Construction of Internet Portals with Machine LearningInformation Retrieval Journal, 2000