P-DRL: A Framework for Multi-UAVs Dynamic Formation Control under Operational Uncertainty and Unknown Environment
Open Access
Abstract
Unmanned aerial vehicle (UAV) formation flying is an efficient and economical operation mode for air transportation systems. To improve the effectiveness of synergetic formation control for UAVs, this paper proposes a pairwise conflict resolution approach for UAV formation through mathematical analysis and designs a dynamic pairing and deep reinforcement learning framework (P-DRL formation control framework). Firstly, a new pairwise UAV formation control theorem is proposed, which breaks down the multi-UAVs formation control problem into multiple sequential control problems involving UAV pairs through a dynamic pairing algorithm. The training difficulty of Agents that only control each pair (two UAVs) is lower compared to controlling all UAVs directly, resulting in better and more stable formation control performance. Then, a deep reinforcement learning model for a UAV pair based on the Environment–Agent interaction is built, where segmented reward functions are designed to reduce the collision possibility of UAVs. Finally, P-DRL completes the formation control task of the UAV fleet through continuous pairing and Agent-based pairwise formation control. The simulations used the dynamic pairing algorithm combined with the DRL architectures of asynchronous advantage actor–critic (P-A3C), actor–critic (P-AC), and double deep q-value network (P-DDQN) to achieve synergetic formation control. This approach yielded effective control results with a strong generalization ability. The success rate of controlling dense, fast, and multi-UAV (10–20) formations reached 96.3%, with good real-time performance (17.14 Hz).Keywords
Funding Information
- National Social Science Foundation of China (22&ZD169, U2333214)
- National Natural Science Foundation of China Grant (22&ZD169, U2333214)
This publication has 39 references indexed in Scilit:
- Hierarchical mission replanning for multiple UAV formations performing tasks in dynamic situationComputer Communications, 2023
- UAV target following in complex occluded environments with adaptive multi-modal fusionApplied Intelligence, 2022
- Design and Verification of UAV Formation Controller based on Leader-Follower Method2022
- Autonomous Obstacle Avoidance and Target Tracking of UAV Based on Deep Reinforcement LearningJournal of Intelligent & Robotic Systems, 2022
- Cooperative pursuit of unauthorized UAVs in urban airspace via Multi-agent reinforcement learningTransportation Research Part C: Emerging Technologies, 2021
- A taxonomy of validation strategies to ensure the safe operation of highly automated vehiclesJournal of Intelligent Transportation Systems, 2020
- Cooperative formation control of multiple aerial vehicles based on guidance route in a complex task environmentChinese Journal of Aeronautics, 2020
- Control of connected and automated vehicles: State of the art and future challengesAnnual Reviews in Control, 2018
- Multiple unmanned aerial vehicle autonomous formation via wolf packs mechanism2016
- Design and simulation of fault tolerant flight control based on a physical approachAerospace Science and Technology, 2012