Deep Reinforcement Learning for Multi-Objective Optimization of Latency-Constrained V2X Communications
Open Access
Abstract
Meeting the stringent latency requirements of safety-critical vehicular services is a major challenge for beyond fifth generation (B5G) systems, and rapidly varying channel state information (CSI) in device-to-device (D2D) enabled vehicle-to-everything (V2X) communication makes the problem harder. This paper addresses power allocation in a D2D-enabled vehicular network in which vehicle-to-vehicle (V2V) links reuse the spectrum of vehicle-to-infrastructure (V2I) links. The problem is formulated as a non-convex optimization problem that maximizes the V2I sum capacity subject to a V2V latency constraint. We use deep reinforcement learning to jointly select the V2I and V2V transmit powers from large-scale channel information, trained from measured outcomes rather than from the closed-form channel expressions used by the optimization-based solution, and a normalized preference weight that trades V2I capacity against V2V latency within a single network. The framework is implemented with a discrete-action deep Q-network (DQN) and with a continuous-action deep deterministic policy gradient (DDPG) agent. Under the nominal channel model, the DQN attains 85–95% and the DDPG agent 90–99% of the sum capacity of the model-based optimum, with a per-block computation time two to three orders of magnitude lower. When the reported large-scale channel gains contain errors of 2 dB, the model-based solution violates the V2V latency constraint on 46–56% of the links, whereas agents trained under such errors keep the violation rate below 0.5%. DDPG is preferable when the channel model is accurate, whereas the DQN trains more reliably under CSI errors. Queue-level simulations further show that, at the considered traffic loads, queueing rather than transmission failures dominates the delay tail.Keywords
Funding Information
- ict division
- Government of the People’s Republic of Bangladesh
- University of the West of Scotland
This publication has 40 references indexed in Scilit:
- Radio Resource Allocation in 5G-NR V2X: A Multi-Agent Actor-Critic Based ApproachIEEE Access, 2023
- A Survey of Scheduling in 5G URLLC and Outlook for Emerging 6G SystemsIEEE Access, 2023
- A practical guide to multi-objective reinforcement learning and planningAutonomous Agents and Multi-Agent Systems, 2022
- A Tutorial on 5G NR V2X CommunicationsIEEE Communications Surveys & Tutorials, 2021
- Deep-Reinforcement-Learning-Based Mode Selection and Resource Allocation for Cellular V2X CommunicationsIEEE Internet of Things Journal, 2019
- Spectrum Sharing in Vehicular Networks Based on Multi-Agent Reinforcement LearningIEEE Journal on Selected Areas in Communications, 2019
- Resource Allocation for Low-Latency Vehicular Communications: An Effective Capacity PerspectiveIEEE Journal on Selected Areas in Communications, 2019
- Dynamic Resource Allocation for Optimized Latency and Reliability in Vehicular NetworksIEEE Access, 2018
- Toward Massive, Ultrareliable, and Low-Latency Wireless Communication With Short PacketsProceedings of the IEEE, 2016
- Human-level control through deep reinforcement learningNature, 2015