STAGE: LLM-Driven Semantic and Topological Augmented Graph Embedding for Text-Attributed Graphs
Open Access
- 6 May 2026
- journal article
- research article
- Published by MDPI AG in Mathematics
- Vol. 14 (9) , 1568
Abstract
Text-attributed graphs (TAGs) require models to jointly exploit node text and graph structure, yet doing so effectively remains difficult when node text is sparse and the structural context is large. Here, we propose STAGE (Semantic and Topological Augmented Graph Embedding), a two-stage framework for representation learning on TAGs. In Stage I, a frozen large language model is used offline to generate explanatory text that enriches compressed node attributes without introducing online LLM training cost. In Stage II, STAGE performs structure-aware representation learning under a fixed global token budget by combining random-walk-based structural context with graph-conditioned token reduction before PLM encoding. This design preserves informative semantic content while preventing unconstrained sequence expansion. Experiments on seven benchmark datasets show that STAGE consistently outperforms strong baselines under the same evaluation setting and maintains favorable efficiency under bounded input-length constraints.Keywords
Funding Information
- National Natural Science Foundation of China (62576301)
- Xiamen Municipal Bureau of Science and Technology (3502Z20241027)
- State Key Laboratory of Multimodal Artificial Intelligence Systems (MAIS2024101)
This publication has 18 references indexed in Scilit:
- Structure-aware transformer for enhanced low-resolution human pose estimationThe Visual Computer, 2025
- Taco-DDI: accurate prediction of drug-drug interaction events using graph transformer-based architecture and dynamic co-attention matricesNeural Networks, 2025
- Leveraging large language model as news sentiment predictor in stock markets: a knowledge-enhanced strategyDiscover Computing, 2025
- Health Insurance Fraud Detection via Multiview Heterogeneous Information Networks With Augmented Graph Structure LearningIEEE Transactions on Computational Social Systems, 2024
- Global-local graph attention: unifying global and local attention for node classificationThe Computer Journal, 2024
- Spatio-temporal fusion graph convolutional network for traffic flow forecastingInformation Fusion, 2023
- Learned Token Pruning for Transformers2022
- Stacked Capsule Graph Autoencoders for geometry-aware 3D head pose estimationComputer Vision and Image Understanding, 2021
- Multi-label feature selection with constraint regression and adaptive spectral graphKnowledge-Based Systems, 2021
- Unsupervised spectral feature selection with l1-norm graphNeurocomputing, 2016