Beyond-Visual-Range Air Combat Tactics Auto-Generation by Reinforcement Learning
Authors:
- 1 July 2020
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 1-8
Abstract
For quite a long time, effective Beyond-Visual-Range (BVR) air combat tactics can only be discovered by human pilots in the actual combat process. However, due to the lack of actual combat opportunities, making new air combat tactics innovation was generally considered quite difficult. To address this challenge, we first introduced a solely end-to-end Reinforcement Learning (RL) approach for training competitive air combat agents with adversarial self-play from scratch in a high fidelity air combat simulation environment during training. Furthermore, a Key Air Combat Event Reward Shaping (KAERS) mechanism was proposed to provide sparse but objective shaped rewards beyond episodic win/lose signal to accelerate the initial machine learning process. Experimental results showed that multiple valuable air combat tactical behaviors emerged progressively. We hope this study could be extended to the future of air combat machine intelligence research.Keywords
This publication has 15 references indexed in Scilit:
- Mastering the game of Go without human knowledgeNature, 2017
- Mastering the game of Go with deep neural networks and tree searchNature, 2016
- Human-level control through deep reinforcement learningNature, 2015
- Self-organizing neural networks for learning air combat maneuvers2012
- Air-Combat Strategy Using Approximate Dynamic ProgrammingJournal of Guidance, Control, and Dynamics, 2010
- Modeling Air Combat by a Moving Horizon Influence Diagram GameJournal of Guidance, Control, and Dynamics, 2006
- Generalised weakened fictitious playGames and Economic Behavior, 2006
- Deep BlueArtificial Intelligence, 2002
- Markov games as a framework for multi-agent reinforcement learning1994
- Development of a tactical guidance research and evaluation system (TGRES)1989