Deep Multi-Agent Reinforcement Learning With Minimal Cross-Agent Communication for SFC Partitioning
Open Access
- 24 April 2023
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 11, 40384-40398
Abstract
Network Function Virtualization (NFV) decouples network functions from the underlying specialized devices, enabling network processing with higher flexibility and resource efficiency. This promotes the use of virtual network functions (VNFs), which can be grouped to form a service function chain (SFC). A critical challenge in NFV is SFC partitioning (SFCP), which is mathematically expressed as a graph-to-graph mapping problem. Given its NP-hardness, SFCP is commonly solved by approximation methods. Yet, the relevant literature exhibits a gradual shift towards data-driven SFCP frameworks, such as (deep) reinforcement learning (RL). In this article, we initially identify crucial limitations of existing RL-based SFCP approaches. In particular, we argue that most of them stem from the centralized implementation of RL schemes. Therefore, we devise a cooperative deep multi-agent reinforcement learning (DMARL) scheme for decentralized SFCP, which fosters the efficient communication of neighboring agents. Our simulation results (i) demonstrate that DMARL outperforms a state-of-the-art centralized double deep Q-learning algorithm, (ii) unfold the fundamental behaviors learned by the team of agents, (iii) highlight the importance of information exchange between agents, and (iv) showcase the implications stemming from various network topologies on the DMARL efficiency.Keywords
Funding Information
- European Union’s Horizon 2020 Research and Innovation Program (101017109 (DAEMON))
This publication has 22 references indexed in Scilit:
- Optimal VNF Placement via Deep Reinforcement Learning in SDN/NFV-Enabled NetworksIEEE Journal on Selected Areas in Communications, 2019
- A Deep Reinforcement Learning Approach for VNF Forwarding Graph EmbeddingIEEE Transactions on Network and Service Management, 2019
- NFVdeep2019
- Multi-domain non-cooperative VNF-FG embedding: A deep reinforcement learning approach2019
- VNF placement with replication for Loac balancing in NFV networks2017
- Virtual Network Embedding via Monte Carlo Tree SearchIEEE Transactions on Cybernetics, 2017
- Multi-Provider Service Chain Embedding With NestorIEEE Transactions on Network and Service Management, 2017
- Human-level control through deep reinforcement learningNature, 2015
- Independent reinforcement learners in cooperative Markov games: a survey regarding coordination problemsThe Knowledge Engineering Review, 2012
- Multi-Agent Reinforcement Learning: Independent vs. Cooperative Agents1993