Unsupervised Universal Self-Attention Network for Graph Classification
Authors:
- Dai Quoc NguyenDNDai Quoc Nguyen
- Dinh PhungDPDinh Phung
Preprint
Abstract
Existing graph neural network-based models often have weaknesses in exploiting potential dependencies among nodes and graph structure properties. To this end, we present U2GNN, a novel embedding model leveraging on the strength of the recently introduced universal self-attention network (Dehghani et al., 2019), to learn low-dimensional embeddings of graphs which can be used for graph classification. In particular, given an input graph, U2GNN first applies a self-attention computation, which is then followed by a recurrent transition to iteratively memorize its attention on vector representations of each node and its neighbors across each iteration. Thus, U2GNN can address the weaknesses in the existing models in order to produce plausible node embeddings whose sum is the final embedding of the whole graph. Experimental results in both supervised and unsupervised training settings show that our U2GNN produces new state-of-the-art performances on a range of well-known benchmark datasets for the graph classification task. To the best of our knowledge, this is the first work showing that a unsupervised model can significantly work better than supervised models by a large margin.Keywords
All Related Versions
This publication has 9 references indexed in Scilit:
- Graph Kernels: A SurveyJournal of Artificial Intelligence Research, 2021
- Structured self-attention architecture for graph-level representation learningPattern Recognition, 2020
- A Comprehensive Survey on Graph Neural NetworksIEEE Transactions on Neural Networks and Learning Systems, 2020
- Graph neural networks: A review of methods and applicationsAI Open, 2020
- An End-to-End Deep Learning Architecture for Graph ClassificationProceedings of the AAAI Conference on Artificial Intelligence, 2018
- Deep Residual Learning for Image Recognition2016
- On Using Very Large Target Vocabulary for Neural Machine Translation2015
- Shortest-Path Kernels on Graphs2005
- Structure-activity relationship of mutagenic aromatic and heteroaromatic nitro compounds. Correlation with molecular orbital energies and hydrophobicityJournal of Medicinal Chemistry, 1991