Multi-Agent Deep Reinforcement Learning for Vehicular Resource Allocation: A Comparison Study of Different Agent Cooperation Levels
Authors:
- 24 June 2025
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Vehicular Technology
- Vol. 74 (12) , 18900-18913
Abstract
The spectrum sharing design (such as the underlay sidelink configuration in cellular networks) for vehicle-to-vehicle (V2V) and vehicle-to-network (V2N) links can improve the efficiency of spectrum utilization for vehicular networks. However, differentiated service requirements for V2V and V2N links and the greater complexity of interference management make it challenging for conventional centralized learning-based methods in the face of real-world deployment. This motivates the exploration of multi-agent deep reinforcement learning (DRL) solutions, while existing research has predominantly focused on cooperative setting that causes large communication overheads for real-world deployment. This paper proposed a decentralized resource allocation scheme for joint spectrum sub-band and V2V transmission power selection that utilizes federated learning (FL) to formulate a mixed setup to avoid frequent communication and enable easy model sharing, termed FL-MX for the design of FL and mixed DRL agent cooperation level. Different levels of cooperation among agents are also explored by implementing different state-of-the-art (SOTA) DRL-based resource allocation algorithms on the developed standard-compliant simulation platform. Through extensive simulations, it is observed that the proposed FL-MX manages to converge effectively and can match the best SOTA performance, gaining a 22.55% and 9.5% improvement in the V2V payload delivery ratio and V2N throughput compared to the random baseline. Such results make it promising for future research to investigate multi-agent DRL designs at a mixed agent cooperation level for high-performing network decision solutions with less frequent communication demands.Keywords
This publication has 32 references indexed in Scilit:
- Graph neural network-based cell switching for energy optimization in ultra-dense heterogeneous networksScientific Reports, 2022
- Resource allocation for Vehicle‐to‐Everything communications: A surveyIET Networks, 2022
- Survey on Device to Device (D2D) Communication for 5GB/6G Networks: Concept, Applications, Challenges, and Future DirectionsIEEE Access, 2022
- Multi-Agent Reinforcement Learning: A Selective Overview of Theories and Algorithms2021
- Towards 6G wireless communication networks: vision, enabling technologies, and new paradigm shiftsScience China Information Sciences, 2020
- Array programming with NumPyNature, 2020
- Efficient partition of integer optimization problems with one-hot encodingScientific Reports, 2019
- Human-level control through deep reinforcement learningNature, 2015
- Markov games as a framework for multi-agent reinforcement learning1994
- Self-improving reactive agents based on reinforcement learning, planning and teachingMachine Learning, 1992