Masked Graph Transformer for Large-Scale Recommendation
Open Access
- 10 July 2024
- conference paper
- Published by Association for Computing Machinery (ACM)
- p. 2502-2506
Abstract
Graph Transformers have garnered significant attention for learning graph-structured data, thanks to their superb ability to capture long-range dependencies among nodes. However, the quadratic space and time complexity hinders the scalability of Graph Transformers, particularly for large-scale recommendation. Here we propose an efficient Masked Graph Transformer, named MGFormer, capable of capturing all-pair interactions among nodes with a linear complexity. To achieve this, we treat all user/item nodes as independent tokens, enhance them with positional embeddings, and feed them into a kernelized attention module. Additionally, we incorporate learnable relative degree information to appropriately reweigh the attentions. Experimental results show the superior performance of our MGFormer, even with a single attention layer.Keywords
All Related Versions
This publication has 18 references indexed in Scilit:
- PaCEr: Network Embedding From Positional to Structural2024
- Federated Few-shot Learning2023
- Kernel Ridge Regression-Based Graph Dataset Distillation2023
- Graph Transformer for Recommendation2023
- Improving Fairness in Graph Neural Networks via Mitigating Sensitive Attribute Leakage2022
- Global Self-Attention as a Replacement for Graph Convolution2022
- Towards Representation Alignment and Uniformity in Collaborative Filtering2022
- Graph Neural Transport Networks with Non-local Attentions for Recommender Systems2022
- MixGCF2021
- LightGCN2020