AoI-Aware Resource Allocation for Platoon-Based C-V2X Networks via Multi-Agent Multi-Task Reinforcement Learning
Top Cited Papers
- 20 March 2023
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Vehicular Technology
- Vol. 72 (8) , 9880-9896
Abstract
This paper investigates the problem of age of information (AoI) aware radio resource management for a platooning system. Multiple autonomous platoons exploit the cellular wireless vehicle-to-everything (C-V2X) communication technology to disseminate the cooperative awareness messages (CAMs) to their followers while ensuring timely delivery of safety-critical messages to the Road-Side Unit (RSU). To lower the computational load at the RSU and cope with the challenges of dynamic channel conditions, we exploit a distributed resource allocation framework based on multi-agent reinforcement learning (MARL), where each platoon leader (PL) acts as an agent and interacts with the environment to learn its optimal policy. Motivated by the existing literature in RL, we propose two novel MARL frameworks based on the multi-agent deep deterministic policy gradient (MADDPG), named Modified MADDPG, and Modified MADDPG with task decomposition. Both algorithms train two critics with the following goals: A global critic which estimates the global expected reward and motivates the agents toward a cooperating behavior and an exclusive local critic for each agent that estimates the local individual reward. Furthermore, based on the tasks each agent has to accomplish, in the second algorithm, the holistic individual reward of each agent is decomposed into multiple sub-reward functions where task-wise value functions are learned separately. Numerical results indicate our proposed algorithms' effectiveness compared with other contemporary RL frameworks, e.g., federated reinforcement learning (FRL) in terms of AoI performance and CAM message transmission probability.Keywords
All Related Versions
Funding Information
- Bundesministerium für Forschung, Technologie und Raumfahrt
- Souveran. Digital. Vernetzt. (6G-RIC, 16KISK020 K, 16KISK031)
This publication has 39 references indexed in Scilit:
- Resource Allocation for Cellular-based Inter-Vehicle Communications in Autonomous MultiplatoonsIEEE Transactions on Vehicular Technology, 2017
- Platoons of connected vehicles can double throughput in urban roadsTransportation Research Part C: Emerging Technologies, 2017
- Better Platooning Control Toward Autonomous Driving : An LTE Device-to-Device Communications Strategy That Meets Ultralow Latency RequirementsIEEE Vehicular Technology Magazine, 2017
- LTE-V: A TD-LTE-Based V2X Solution for Future Vehicular NetworkIEEE Internet of Things Journal, 2016
- A Survey on the Coordination of Connected and Automated Vehicles at Intersections and Merging at Highway On-RampsIEEE Transactions on Intelligent Transportation Systems, 2016
- A Survey on Platoon-Based Vehicular Cyber-Physical SystemsIEEE Communications Surveys & Tutorials, 2015
- Data-Driven Intelligent Transportation Systems: A SurveyIEEE Transactions on Intelligent Transportation Systems, 2011
- Minimizing age of information in vehicular networks2011
- Vehicle sorting for platoon formation: Impacts on highway entry and throughputTransportation Research Part C: Emerging Technologies, 2005
- Q-learningMachine Learning, 1992