Hierarchical Decision Making-Based Intelligent Game Confrontation on UAV Swarm
Open Access
- 21 November 2025
- Vol. 12 (12) , 1033
Abstract
To address the challenge of decision making in close-range air combat for fixed-wing unmanned air vehicle (UAV) swarms, this paper proposes a distributed Hierarchical Cooperative Soft Actor-Critic with maximum entropy (HC-SAC) framework. A top-level target decision-making and bottom-level maneuvering framework is designed to resolve convergence issues in traditional multi-agent reinforcement learning, typically for long mission durations and complex state spaces. Friendly tactics are incorporated into top-level decisions to enhance coordination, with both offensive and defensive sub-policies balanced for swarm confrontations. These policies are trained using the Soft Actor-Critic (SAC) deep reinforcement learning algorithm with specific reward functions, and the effectiveness of this method is verified through 5v5 swarm game confrontation simulations.Keywords
This publication has 11 references indexed in Scilit:
- Autonomous Dogfight Decision-Making for Air Combat Based on Reinforcement Learning with Automatic Opponent SamplingAerospace, 2025
- Mastering air combat game with deep reinforcement learningDefence Technology, 2024
- Hierarchical Multi-Agent Reinforcement Learning for Air Combat Maneuvering2023
- Cooperative maneuver decision making for multi-UAV air combat based on incomplete information dynamic gameDefence Technology, 2023
- DARPA OFFSET: A Vision for Advanced Swarm Systems through Agile Technology Development and ExperimentationField Robotics, 2023
- Hierarchical multi‐agent reinforcement learning for multi‐aircraft close‐range air combatIET Control Theory & Applications, 2022
- Hierarchical Reinforcement Learning for Air-to-Air Combat2021
- Hierarchical Reinforcement LearningACM Computing Surveys, 2021
- Multiple Pursuer Multiple Evader Differential GamesIEEE Transactions on Automatic Control, 2020
- An integrated environment for tactical guidance research and evaluation1990