Scalable Cooperative Decision-Making in Multi-UAV Confrontations: An Attention-Based Multiagent Actor–Critic Approach
- 20 May 2025
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Aerospace and Electronic Systems
- Vol. 61 (6) , 15195-15209
Abstract
With the increasing use of unmanned aerial vehicles (UAVs) in military operations, autonomous cooperative decision-making for multiple UAVs in aerial confrontations has become a critical research challenge. This paper presents an attention-based multi-agent actor-critic (AMAAC) algorithm for UAV aerial confrontation decision-making. The algorithm combines multi-head attention and self-play within the centralized training-distributed execution (CTDE) framework, extending the actor-critic approach based on the missile hit probability prediction model (MHPAC) to multi-UAV scenarios. A fighter observation encoder (OFE) and a centralized critic network based on the attention mechanism are introduced to adapt to varying number of UAVs (different scales) and enhance training performance. Additionally, self-play-based extended training is used to generalize offensive and defensive strategies from small-scale aerial confrontations to larger scenarios. Experimental results demonstrate that the AMAAC algorithm achieves superior training effectiveness, and the strategies it produces perform well across various confrontation scales, even beyond the training scenario's scale. Compared to other decision-making algorithms, such as Multi-agent Proximal Policy Optimization (MAPPO), Multi-agent Hierarchical Policy Gradient (MAHPG), and the State-Event-Condition-Action (SECA) algorithm, the AMAAC-trained strategies yield higher win ratios and kill-death ratios in different scenarios, validating the algorithm's effectiveness and scalability.Keywords
Funding Information
- Foundation Enhancement Project for Science and Technology (2021-JCJQ-JJ-081)
- Young Talent Fund of Association for Science and Technology (20220101)
- Postdoctoral International Exchange Project (YJ20220347)
- National Natural Science Foundation of China (62303489)
- Postdoctoral Science Foundation General Program (2022M723877)
- Postdoctoral Science Foundation Special Funding (2023T160790)
This publication has 34 references indexed in Scilit:
- Graph-based multi-agent reinforcement learning for collaborative search and tracking of multiple UAVsChinese Journal of Aeronautics, 2024
- Unit coordination knowledge enhanced autonomous decision-making approach of heterogeneous UAV formationChinese Journal of Aeronautics, 2024
- Enhanced missile hit probability actor-critic algorithm for autonomous decision-making in air-to-air confrontationAerospace Science and Technology, 2024
- Graph-based multi-agent reinforcement learning for large-scale UAVs swarm system controlAerospace Science and Technology, 2024
- Cascade operation-enhanced high-resolution representation learning for meticulous segmentation of bridge cracksAdvanced Engineering Informatics, 2024
- Fault-Tolerant Cooperative Control of Unmanned Aerial Vehicles2024
- A multi-population immune plasma algorithm for path planning of unmanned combat aerial vehicleAdvanced Engineering Informatics, 2022
- STABC-IR: An air target intention recognition method based on bidirectional gated recurrent unit and conditional random field with space-time attention mechanismChinese Journal of Aeronautics, 2022
- An autonomous aerial combat framework for two-on-two engagements based on basic fighter maneuversAerospace Science and Technology, 2018
- Aircraft Control and Simulation: Dynamics, Controls Design, and Autonomous Systems2015