Digital-Twin-Assisted Task Assignment in Multi-UAV Systems: A Deep Reinforcement Learning Approach
Open Access
- 31 March 2023
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Internet of Things Journal
- Vol. 10 (17) , 15362-15375
Abstract
Most existing multi-unmanned aerial vehicle (multi-UAV) systems focus on fly path or energy consumption for task assignment, while little attention has been paid to the dynamic feature of the task, resulting in poor task completion ratio. The machine learning (ML) paradigm provides new methodologies for task assignment. However, ML methods are usually of heavy resource-consumption that cannot be directly applied in the UAV. In this paper, a digital twin (DT) assisted task assignment approach is proposed to improve the resource-intensive utilization and the efficiency of deep reinforcement learning (DRL) in multi-UAV system. The approach has a three-layer network structure which can dynamically assign tasks based on the task time constraints. Moreover, the approach is divided into two stages of initial task-assignment and task-reassignment. In the first stage, airship divides a task into multiple subtasks according to the shortest distance based on genetic algorithm and assigns them to UAVs. In the second stage, the DT can be leveraged to enable the airships to learn from the features of tasks and to generate the Q-value of the estimated value network of DRL for UAVs via pre-train of DT. The Q-value can be directly applied for deep Q-learning network (DQN) in the UAVs to reduce the training episode. Furthermore, the DQN is adopted to train task-reassignment strategy. Simulation results indicate that the DQN with DT can significantly reduce the training episode, improving 30% of the task completion ratio and 19% of the system energy efficiency compared with that of the baseline methods.Keywords
Funding Information
- Guangxi Natural Science Foundation of China (2019GXNSFFA245007)
- National Natural Science Foundation of China (U22A2054, 61971148)
- Guangxi Key Laboratory of Wireless Wideband Communication and Signal Processing (GXKL06200103)
- National Key Research and Development Program of China (2020YFB1807802)
- Research Grant of University of Macau (MYRG2020-00107-IOTSC)
- Guangdong-Macau Joint Laboratory for Advanced and Intelligent Computing (GDST2020B1212030003)
This publication has 30 references indexed in Scilit:
- Joint Deployment and Task Scheduling Optimization for Large-Scale Mobile Users in Multi-UAV-Enabled Mobile Edge ComputingIEEE Transactions on Cybernetics, 2019
- Evolutionary Collaborative Human-UAV Search for Escaped CriminalsIEEE Transactions on Evolutionary Computation, 2019
- Interference Management for Cellular-Connected UAVs: A Deep Reinforcement Learning ApproachIEEE Transactions on Wireless Communications, 2019
- An iterative strategy for task assignment and path planning of distributed multiple unmanned aerial vehiclesAerospace Science and Technology, 2019
- Joint Blocklength and Location Optimization for URLLC-Enabled UAV Relay SystemsIEEE Communications Letters, 2019
- When Mobile Crowd Sensing Meets UAV: Energy-Efficient Task Assignment and Route PlanningIEEE Transactions on Communications, 2018
- Multi-UAV Task Assignment With Parameter and Time-Sensitive Uncertainties Using Modified Two-Part Wolf Pack Search AlgorithmIEEE Transactions on Aerospace and Electronic Systems, 2018
- Multi-UAV reconnaissance task allocation for heterogeneous targets using an opposition-based genetic algorithm with double-chromosome encodingChinese Journal of Aeronautics, 2018
- Multiple UAVs in forest fire fighting mission using particle swarm optimization2017
- Numerical phase correction method for terahertz time-domain reflection spectroscopyJournal of Applied Physics, 2004