Digital twin based DDPG reinforcement learning for sum-rate maximization of AI-UAV communications
Authors:
Open Access
- 18 July 2024
- journal article
- research article
- Published by Springer Nature in EURASIP Journal on Wireless Communications and Networking
- Vol. 2024 (1) , 1-22
Abstract
Construction of wireless infrastructure using unmanned aerial vehicle (UAV) can effectively expand the coverage and support high-density traffic of next-generation communication systems. Designing wireless systems including UAVs as aerial base stations (ABSs) is a challenging task, due to the mobility of ABSs causing time-varying nature of environmental surroundings and relative propagation paths to user equipment (UE) devices. Therefore, it is essential to have an accurate estimate of the channel for varying positioning of the UAVs. In this paper, we propose to adopt a digital twin based performance evaluation procedure for wireless systems including ABSs, providing enhanced accuracy of channel modeling for specific target deployment areas. Using ray-tracing channel models reflecting detailed building and terrain information of the transmission environment, an UAV position optimization algorithm based on reinforcement learning is presented. By utilizing deep deterministic policy gradient (DDPG), the proposed algorithm calculates the overall throughput in the digital twin and determines the desired states of the UAV. Performance evaluation results demonstrate the trajectory training ability of the algorithm and the performance advantage of the system with a reduced amount of shadow area compared to those with ground base stations (GBSs).Keywords
Funding Information
- Seoul Buseniss Agency (CY230079)
- Ministry of SMEs and Startups (RS-2003-00279316)
This publication has 41 references indexed in Scilit:
- Energy-Efficient Cellular Offloading Optimization for UAV-Aided Networks2023
- Evaluation of Beamsteering Performance in MultiuserMIMO Unmanned Aerial Base Stations NetworksIEEE Access, 2022
- Artificial Intelligence‐Enabled Sensing Technologies in the 5G/Internet of Things Era: From Virtual Reality/Augmented Reality to the Digital TwinAdvanced Intelligent Systems, 2022
- Multi-User Hybrid Beamforming System Based on Deep Neural Network in Millimeter-Wave CommunicationIEEE Access, 2020
- UAV Positioning Based on Multi-Sensor FusionIEEE Access, 2020
- Deep Deterministic Policy Gradient (DDPG)-Based Resource Allocation Scheme for NOMA Vehicular CommunicationsIEEE Access, 2020
- A Tutorial on Beam Management for 3GPP NR at mmWave FrequenciesIEEE Communications Surveys & Tutorials, 2018
- Ray Tracing for Radio Propagation Modeling: Principles and ApplicationsIEEE Access, 2015
- The Ornstein–Uhlenbeck process as a model of a low pass filtered white noiseMetrologia, 2008
- Shooting and bouncing rays: calculating the RCS of an arbitrarily shaped cavityIEEE Transactions on Antennas and Propagation, 1989