Uncertainty-Aware Multi-Agent Reinforcement Learning for Anti-Interference Trajectory Planning of Cellular-Connected UAVs
- 4 September 2025
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Vehicular Technology
- Vol. 75 (2) , 2864-2880
Abstract
Cellular-connected unmanned aerial vehicles (C-UAVs) will be an integral component of future wireless networks. Thanks to the mobility and maneuverability of UAVs, we can transform the interference management and route scheduling problems of C-UAVs into an anti-interference trajectory planning problem, aiming to jointly minimize the UAV mission time and transmission outage time. However, none of the existing methods have taken both the spatio-temporal uncertainty of interference sources and multi-UAV trajectory planning into consideration. To address this issue, we propose a novel method, referred to as uncertainty-aware multi-agent reinforcement learning (UA-MARL), for anti-interference trajectory planning of C-UAVs. In UA-MARL, a transmission outage probability (TOP) has been introduced to improve the robustness of the model. A transmission outage probability experience memory (TOPEM) has been designed to increase sample efficiency and reduce inference time. MARL algorithms integrated with an adaptive post-decision state (PDS) have been introduced to accelerate the convergence and stabilize the training. Experimental results show that UA-MARL outperforms baselines in average reward, convergence efficiency, and convergence stability. Furthermore, we find that higher residential density and wider considered area will lead to a decrease in training efficiency and stability.Keywords
Funding Information
- Japan Science and Technology Agency's (JST) Adopting Sustainable Partnerships for Innovative Research Ecosystem (JPMJAP2325)
- Japan Society for the Promotion of Science Grants-in-Aid for Scientific Research (JP24K02937)
- JST SPRING GX (JPMJSP2108)
This publication has 41 references indexed in Scilit:
- UAV Path Planning Based on the Average TD3 Algorithm With Prioritized Experience ReplayIEEE Access, 2024
- A 3D Path Planning Algorithm for the UAV Based on the Improved Double-DqnAlgorithm2023
- A UAV Path Planning Method in Three-Dimensional Urban Airspace based on Safe Reinforcement Learning2023
- Autonomous UAV Path Planning Using Modified PSO for UAV-Assisted Wireless NetworksIEEE Access, 2023
- Coverage Path Planning Optimization of Heterogeneous UAVs Group for Precision AgricultureIEEE Access, 2023
- Joint Trajectory and Power Control Design for UAV Anti-Jamming Communication Network2022
- Energy-Optimal Flight Strategy for Solar-Powered Aircraft Using Reinforcement Learning With Discrete ActionsIEEE Access, 2021
- A New Anti-interference Fitering Algorithm for Quad Rotor UAV2021
- Packages delivery based on marker detection for UAVs2020
- Multi-Agent Deep Reinforcement Learning for Trajectory Design and Power Allocation in Multi-UAV NetworksIEEE Access, 2020