Towards Expansive and Adaptive Hard Negative Mining: Graph Contrastive Learning via Subspace Preserving
Open Access
- 13 May 2024
- conference paper
- Published by Association for Computing Machinery (ACM)
- p. 322-333
Abstract
Graph Neural Networks (GNNs) have emerged as the predominant approach for analyzing graph data on the web and beyond. Contrastive learning (CL), a self-supervised paradigm, not only mitigates reliance on annotations but also has potential in performance. The hard negative sampling strategy that benefits CL in other domains proves ineffective in the context of Graph Contrastive Learning (GCL) due to the message passing mechanism. Embracing the subspace hypothesis in clustering, we propose a method towards expansive and adaptive hard negative mining, referred to as G raph contR astive leA rning via subsP ace prE serving (GRAPE ). Beyond homophily, we argue that false negatives are prevalent over an expansive range and exploring them confers benefits upon GCL. Diverging from existing neighbor-based methods, our method seeks to mine long-range hard negatives throughout subspace, where message passing is conceived as interactions between subspaces. %Empirical investigations back up this strategy. Additionally, our method adaptively scales the hard negatives set through subspace preservation during training. In practice, we develop two schemes to enhance GCL that are pluggable into existing GCL frameworks. The underlying mechanisms are analyzed and the connections to related methods are investigated. Comprehensive experiments demonstrate that our method outperforms across diverse graph datasets and remains competitive across varied application scenarios\footnoteOur code is available at https://github.com/zz-haooo/WWW24-GRAPE. .Keywords
This publication has 49 references indexed in Scilit:
- Exploring Simple Siamese Representation Learning2021
- Understanding the Behaviour of Contrastive Loss2021
- Learning a Self-Expressive Network for Subspace Clustering2021
- End-to-End Learning of Visual Representations From Uncurated Instructional Videos2020
- Deep Low-Rank Subspace Clustering2020
- Oracle Based Active Set Algorithm for Scalable Elastic Net Subspace Clustering2016
- Smooth Representation Clustering2014
- Robust and Efficient Subspace Segmentation via Least Squares Regression2012
- Eigenfaces vs. Fisherfaces: recognition using class specific linear projectionIEEE Transactions on Pattern Analysis and Machine Intelligence, 1997
- Feature selection: evaluation, application, and small sample performanceIEEE Transactions on Pattern Analysis and Machine Intelligence, 1997