Universal Graph Transformer Self-Attention Networks
Authors:
Open Access
- 25 April 2022
- conference paper
- Published by Association for Computing Machinery (ACM)
- p. 193-196
Abstract
We introduce a transformer-based GNN model, named UGformer, to learn graph representations. In particular, we present two UGformer variants, wherein the first variant (publicized in September 2019) is to leverage the transformer on a set of sampled neighbors for each input node, while the second (publicized in May 2021) is to leverage the transformer on all input nodes. Experimental results demonstrate that the first UGformer variant achieves state-of-the-art accuracies on benchmark datasets for graph classification in both inductive setting and unsupervised transductive setting; and the second UGformer variant obtains state-of-the-art accuracies for inductive text classification. The code is available at: https://github.com/daiquocnguyen/Graph-Transformer.Keywords
All Related Versions
This publication has 8 references indexed in Scilit:
- ReGVD: Revisiting Graph Neural Networks for Vulnerability Detection2022
- Node Co-occurrence based Graph Neural Networks for Knowledge Graph Link Prediction2022
- Structured self-attention architecture for graph-level representation learningPattern Recognition, 2020
- Every Document Owns Its Structure: Inductive Text Classification via Graph Neural Networks2020
- Graph Convolutional Networks for Text ClassificationProceedings of the AAAI Conference on Artificial Intelligence, 2019
- An End-to-End Deep Learning Architecture for Graph ClassificationProceedings of the AAAI Conference on Artificial Intelligence, 2018
- Glove: Global Vectors for Word Representation2014
- The Graph Neural Network ModelIEEE Transactions on Neural Networks, 2008