Realizing Midcourse Penetration With Deep Reinforcement Learning
Authors:
Open Access
- 22 June 2021
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 9, 89812-89822
Abstract
A midcourse maneuver controller is obtained using deep reinforcement learning to maintain the survivability of a ballistic missile. First, the midcourse is abstracted as a Markov decision process (MDP) with an unknown system state equation. Then, a controller formed by the Dueling Double Deep Q (D3Q) neural network is used to approximate the state-action value function of the MDP. In order to make the controller’s intelligence improved by deep reinforcement learning, the state space, action space, and instant reward function of the MDP are customized. The controller uses a real-time situation as input and outputs the ignition states of pulse motors. Offline training shows that deep reinforcement learning can achieve the optimal strategy’s convergence after approximately 65 hours. Online tests demonstrate the controller’s ability to avoid an interceptor intelligently and to account for an entry error. In scenarios with multiple random factors, the controller achieved a penetration probability of 100% and a mean re-entry error of less than 5000 m.Keywords
Funding Information
- Chinese Aeronautical Establishment (201929052002)
This publication has 19 references indexed in Scilit:
- Multi-model soft switching tracking control for near-space interceptor based on the disturbance observerProceedings of the Institution of Mechanical Engineers, Part I: Journal of Systems and Control Engineering, 2016
- Nonlinear Differential Games-Based Impact-Angle-Constrained Guidance LawJournal of Guidance, Control, and Dynamics, 2015
- Deep Neural Networks for Acoustic Modeling in Speech Recognition: The Shared Views of Four Research GroupsIEEE Signal Processing Magazine, 2012
- Modeling and Simulation System Design of Tactical Ballistic Missile Interception Based on UML2012
- Markov Decision Processes: A Tool for Sequential Decision Making under UncertaintyMedical Decision Making, 2009
- Three-dimensional midcourse guidance using neural networks for interception of ballistic targetsIEEE Transactions on Aerospace and Electronic Systems, 2002
- Reinforcement Learning: An IntroductionIEEE Transactions on Neural Networks, 1998
- Near-Optimal Guidance Law for Ballistic Missile InterceptionJournal of Guidance, Control, and Dynamics, 1997
- Fighter evasive maneuvers against proportional navigation missileJournal of Aircraft, 1986
- The effects of non‐linear kinematics in optimal evasionOptimal Control Applications and Methods, 1983