Abstract
Graph Contrastive Learning (GCL) stands as a potent framework for unsupervised graph representation learning that has gained traction across numerous graph learning applications. The effectiveness of GCL relies on generating high-quality contrasting samples, enhancing the model's ability to discern graph semantics. However, the prevailing GCL methods face two key challenges: 1) introducing noise during graph augmentations and 2) requiring additional storage for generated samples, which degrade the model performance. In this paper, we propose novel approaches, GKCL (i.e., Graph Knowledge Contrastive Learning) and DGKCL (i.e., Distilled Graph Knowledge Contrastive Learning), that leverage multilevel graph knowledge to create noise-free contrasting pairs. This framework not only addresses the noise-related challenges but also circumvents excessive storage demands. Furthermore, our method incorporates a knowledge distillation component to optimize the trained embedding tables, reducing the model's scale while ensuring superior performance, particularly for the scenarios with smaller embedding sizes. Comprehensive experimental evaluations on three public benchmark datasets underscore the merits of our proposed method and elucidate its properties, which primarily reflect the performance of the proposed method equipped with different embedding sizes and how the distillation weight affects the overall performance
Funding Information
  • National Natural Science Foundation of China (62072257)
  • Australian Research Council (DP22010371, LE220100078)
  • Hong Kong Research Grants Council (15200023)
  • Research Impact Fund (R1015-23)
  • APRC - CityU New Research Initiatives (9610565)
  • New Faculty of CityU
  • CityU - HKIDS Early Career Research (9360163)
  • Hong Kong ITC Innovation and Technology Fund Midstream Research Programme for Universities (ITS/034/22MS)
  • Hong Kong Environmental and Conservation Fund (88/2022)
  • SIRG - CityU Strategic Interdisciplinary Research (7020046)
  • Huawei Innovation Research Program
  • CCF-Tencent Open Fund, Tencent Rhino-Bird Focused Research Program
  • CCF-Ant Research Fund, Ant Group Research Fund
  • CCF-Alimama Tech Kangaroo Fund (2024002)
  • CCF-Bai Chuan-Ebtech Foundation Model Fund
  • Kuaishou