Hierarchical Multi-Agent Reinforcement Learning for Air Combat Maneuvering
Authors:
- 15 December 2023
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 1031-1038
Abstract
The application of artificial intelligence to simulate air-to-air combat scenarios is attracting increasing attention. To date the high-dimensional state and action spaces, the high complexity of situation information (such as imperfect and filtered information, stochasticity, incomplete knowledge about mission targets) and the nonlinear flight dynamics pose significant challenges for accurate air combat decision-making. These challenges are exacerbated when multiple heterogeneous agents are involved. We propose a hierarchical multi-agent reinforcement learning framework for air-to-air combat with multiple heterogeneous agents. In our framework, the decision-making process is divided into two stages of abstraction, where heterogeneous low-level policies control the action of individual units, and a high-level commander policy issues macro commands given the overall mission targets. Low-level policies are trained for accurate unit combat control. Their training is organized in a learning curriculum with increasingly complex training scenarios and league-based self-play. The commander policy is trained on mission targets given pre-trained low-level policies. The empirical validation advocates the advantages of our design choices.Keywords
All Related Versions
This publication has 42 references indexed in Scilit:
- Situational continuity-based air combat autonomous maneuvering decision-makingDefence Technology, 2023
- Short-range air combat maneuver decision of UAV swarm based on multi-agent Transformer introducing virtual objectsEngineering Applications of Artificial Intelligence, 2023
- Autonomous maneuver strategy of swarm air combat based on DDPGAutonomous Intelligent Systems, 2021
- Multi-agent deep reinforcement learning: a surveyArtificial Intelligence Review, 2021
- Cooperative Multi-agent Control Using Deep Reinforcement Learning2017
- The Fighter Aircraft LCS: A Real-World, Machine Innovation Application2004
- Discovering Novel Fighter Combat Maneuvers2002
- Two-Sided, Genetics-Based Learning to Discover Novel Fighter Combat ManeuversLecture Notes in Computer Science, 2001
- The Fighter Aircraft LCS: A Case of Different LCS Goals and Techniques2000
- A differential game in three dimensions: The aerial dogfight scenarioDynamics and Control, 1992