Grace: Toward Routing in Dynamic Network Environments With Graph Embedding
- 11 July 2025
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Networking
- Vol. 33 (6) , 3378-3390
Abstract
Recent efforts have explored adaptive routing via deep reinforcement learning (DRL) techniques without handcrafted parameter engineering. Intrinsically, routing decision-making is essentially a process used to find a subgraph in a graph-structured network. However, previous works seldom took topological relationships into consideration when providing adaptive routing algorithms, causing them to suffer from suboptimal routes in dynamic network environments involving both varying traffic loads and burst traffic. In this paper, we present Grace, a novel graph embedding-based Deep Reinforcement Learning framework tailored for distributed routing algorithm optimization within the Software-Defined Networking (SDN) paradigm. Specifically, Grace leverages graph embedding to translate graph-structured entities into low-dimensional vectors, thereby enabling multiple DRL agents to learn optimal routing paths under dynamic network environments. Unfortunately, training multiple agents encounters inherent challenges in complicated and dynamic network scenarios. In response, we design an adaptive incremental training method for Grace that makes the model adapt to task complexity in a gradual manner, while speeding up its retraining efforts when environments change. To further accelerate convergence, we integrate intrinsic curiosity into Grace to tackle large environments with sparse rewards. Extensive experiments conducted on two real-world topologies demonstrate the rationality and effectiveness of Grace, and the results show throughput improvements of up to 40.1% compared to other state-of-the-art DRL routing algorithms under bursty traffic conditions.Keywords
Funding Information
- National Natural Science Foundation of China (62102302)
- National Key Research and Development Program of China (2025YFB3003200)
- Science and Technology Innovation Team Program of Shaanxi Province (2025RSCXTD-004)
This publication has 29 references indexed in Scilit:
- Federated Traffic Engineering with Supervised Learning in Multi-region Networks2021
- DRL-R: Deep reinforcement learning approach for intelligent routing in software-defined data-center networksJournal of Network and Computer Applications, 2021
- A Multi-agent Reinforcement Learning Perspective on Distributed Traffic Engineering2020
- Applications of Deep Reinforcement Learning in Communications and Networking: A SurveyIEEE Communications Surveys & Tutorials, 2019
- A Survey of Machine Learning Techniques Applied to Software Defined Networking (SDN): Research Issues and ChallengesIEEE Communications Surveys & Tutorials, 2018
- Graph embedding techniques, applications, and performance: A surveyKnowledge-Based Systems, 2018
- Curiosity-Driven Exploration by Self-Supervised Prediction2017
- Building machines that learn and think like peopleBehavioral and Brain Sciences, 2016
- Optimization of The Shortest-Path Routing with Equal-Cost Multi-Path Load Balancing2006
- Another adaptive distributed shortest path algorithmIEEE Transactions on Communications, 1991