Joint UAV Trajectory and RadCom Task Schedule for IVNs: A Game-Embedding Multi-Agent Deep Reinforcement Learning Approach
- 8 November 2024
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Wireless Communications
- Vol. 24 (1) , 181-196
Abstract
Integrated sensing and communication (ISAC) technology has been envisioned to revolutionize the future intelligent vehicle networks (IVNs). Recently, due to the mobility and flexible deployment, unmanned aerial vehicle (UAV) has been regarded as a promising aerial ISAC platform in future IVNs. In this paper, comprehensively considering all the performance on the throughput, sensing accuracy, and sensing rate, we propose a Multi-Agent joint Trajectory control and RadCom task schedule (MA-TRC) scheme for the ISAC-UAV assisted IVN. To make UAVs achieve the optimal decisions autonomously and adaptively, we propose a Game-Embedding Multi-Agent Deep Reinforcement Learning (GE-MADRL) approach. Specifically, considering the complex action space with both discrete and continuous decision variables of UAVs, we develop a multi-agent Parametrized deep Q-network (MAPDQN) based solution, which can help UAVs learn the dynamic and uncertain environment to adaptively obtain the MA-TRC scheme. Furthermore, since UAVs work in a distributed decision making manner, the potential conflicting decisions will impact the network performance. To avoid the decision conflicts among UAVs during the network parameter training, a distributed two-stage Game method is designed as an action adjuster embedded in MAPDQN, by which the learning convergence performance will be further improved and the strategy conflicts can be avoided.Keywords
Funding Information
- National Natural Science Foundation of China (62271100)
- Science and Technology Program of Liaoning Province (2023JH2/101700366)
- Fundamental Research Funds for the Central Universities (DUT24ZD127)
- Xiaomi Young Talents Program
- National Natural Science Foundation of China (62222101, 62071268)
- Aeronautical Science Foundation of China (2022Z071051013)
- Beijing Natural Science Foundation (L232043, L222039)
- Fundamental Research Funds for the Central Universities
This publication has 56 references indexed in Scilit:
- Enabling Joint Communication and Radar Sensing in Mobile Networks—A SurveyIEEE Communications Surveys & Tutorials, 2021
- Rate Splitting Multiple Access for Joint Communication and Sensing Systems with Unmanned Aerial Vehicles2021
- Radio Resource Management in Joint Radar and Communication: A Comprehensive SurveyIEEE Communications Surveys & Tutorials, 2021
- Deep Reinforcement Learning for Fresh Data Collection in UAV-assisted IoT Networks2020
- A UAV-Assisted Data Collection for Wireless Sensor Networks: Autonomous Navigation and SchedulingIEEE Access, 2020
- Effective Capacity Based Power Allocation for the Coexistence of an Integrated Radar and Communication System and a Commercial Communication SystemIEEE Access, 2020
- Joint UAV Hovering Altitude and Power Control for Space-Air-Ground IoT NetworksIEEE Internet of Things Journal, 2018
- Adaptive Wireless Communications2013
- Detection, Estimation, and Modulation Theory, Part I2001
- Von Neumann-Morgenstern solutions to cooperative games without side paymentsBulletin of the American Mathematical Society, 1960