Deep Reinforcement Learning Based Computation Offloading and Trajectory Planning for Multi-UAV Cooperative Target Search
Top Cited Papers
- 15 December 2022
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Journal on Selected Areas in Communications
- Vol. 41 (2) , 504-520
Abstract
Unmanned aerial vehicles (UAVs) are widely used for surveillance and monitoring to complete target search tasks. However, the short battery life and moderate computational capability hinder UAVs to process computation-intensive tasks. The emerging edge computing technologies can alleviate this problem by offloading tasks to the ground edge servers. How to evaluate the search process so as to make optimal offloading decisions and make optimal flying trajectories represent fundamental research challenges. In this paper, we propose to utilize the concept of uncertainty to evaluate the search process, which reflects the reliability of the target search results. Thereafter, we propose a deep reinforcement learning (DRL) technique to jointly make optimal computation offloading decisions and flying orientation choices for multi-UAV cooperative target search. Specifically, we first formulate an uncertainty minimization problem based on the established system model. By introducing a reward function, we prove that the uncertainty minimization problem is equivalent to a reward maximization problem, which is further analyzed by a Markov decision process (MDP). To obtain the optimal task offloading decisions and flying orientation choices, a deep Q-network (DQN) based DRL architecture with a separated Q-network is then proposed. Finally, extensive simulations validate the effectiveness of the proposed techniques, and comprehensive discussions on how different parameters affect the search performance are given.Keywords
Funding Information
- National Natural Science Foundation of China (62101463)
- Natural Science Foundation of Sichuan Province of China (2022NSFSC0863)
- Key Research and Development Program of Sichuan (23GJHZ0209, 23ZHSF0170)
- Fundamental Research Funds for the Central Universities (2682021CX044)
- NSFC (61731017, 62020106001)
This publication has 52 references indexed in Scilit:
- UAV-Empowered Edge Computing Environment for Cyber-Threat Detection in Smart VehiclesIEEE Network, 2018
- Cooperative search-attack mission planning for multi-UAV based on intelligent self-organized algorithmAerospace Science and Technology, 2018
- Multiple Moving Targets Surveillance Based on a Cooperative Network for Multi-UAVIEEE Communications Magazine, 2018
- Energy-Efficient Industrial Internet of UAVs for Power Line Inspection in Smart GridIEEE Transactions on Industrial Informatics, 2018
- Efficient Multi-User Computation Offloading for Mobile-Edge Cloud ComputingIEEE/ACM Transactions on Networking, 2015
- Human-level control through deep reinforcement learningNature, 2015
- Cooperative Search by UAV Teams: A Model Predictive Approach using Dynamic GraphsIEEE Transactions on Aerospace and Electronic Systems, 2011
- Probabilistic target detection by camera-equipped UAVs2010
- Multi-UAV Cooperative Search Using an Opportunistic Learning MethodJournal of Dynamic Systems, Measurement, and Control, 2007
- Flight Performance of Fixed and Rotary Wing Aircraft2006