Minghui Zhao
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Research Subjects
Minghui Zhao has a total of 11 co-authors. Here are the top co-authors by number of shared publications:
- Xuebo Zhang
Institute of Robotics and Automatic Information Systems and Tianjin Key Laboratory of Intelligent Robotics, Nankai University, Tianjin, China
4 shared publications - Chenxu Qian
Institute of Robotics and Automatic Information System, College of Artificial Intelligence and Tianjin Key Laboratory of Intelligent Robotics, Nankai University, Tianjin, China
3 shared publications Jiayi Zhang
Nankai University
1 shared publication- Jiayi Zhang
Institute of Robotics and Automatic Information System, College of Artificial Intelligence, Nankai University, Tianjin 300350, China
1 shared publication - Kuihua Huang
College of System Engineering, National University of Defense Technology, Changsha, China
1 shared publication Lun Li
Institute of Robotics and Automatic Information System, College of Artificial Intelligence, and the Tianjin Key Laboratory of Intelligent Robotics, Nankai University, Tianjin, China
1 shared publication
Latest Publications
- 2 July 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Cybernetics
- Vol. 55 (9) , 4145-4157
A Partial Joint Optimization Algorithm for Autonomous Air Combat Based on Hierarchical Reinforcement Learning
Designing intelligent game strategies for autonomous air combat has suffered from the vast exploration space, lengthy decision-making process, and sparse rewards. Some existing approaches adopt the hierarchical framework to improve the exploration efficiency. However, in these methods, agents in different layers are typically trained independently and operate at fixed frequencies, which limits their performance and hampers their ability to respond to highly dynamic combat situations. In view of this, we present PJOH-TED2, a partial-joint-optimization-based hierarchical (PJOH) learning framework with a time-event dual-driven (TED2) mechanism, for one-on-one beyond-visual-range (BVR) air combat. Specifically, the PJOH learning framework embeds the partial joint optimization mechanism into hierarchical reinforcement learning (HRL), thus improving the exploration efficiency dramatically while enhancing the integration across hierarchical levels. Moreover, the TED2 mechanism combines the advantages of event-driven and time-driven methods, which promote the dynamic response speed of agents as well as avoid redundant actions. In addition, we evaluated this work through a series of games against the state-of-the-art (SOTA) methods in a high-fidelity air combat simulation environment. The results empirically demonstrate that the proposed approach outperforms four SOTA methods with a win rate of at least 71%. Finally, this approach achieved the 1st place in learning methods in the intelligent air game algorithm challenge (IAGAC) by the Chinese Institute of Command and Control among 43 teams.
- 2 July 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Cybernetics
- Vol. 55 (9) , 4145-4157
A Partial Joint Optimization Algorithm for Autonomous Air Combat Based on Hierarchical Reinforcement Learning
Designing intelligent game strategies for autonomous air combat has suffered from the vast exploration space, lengthy decision-making process, and sparse rewards. Some existing approaches adopt the hierarchical framework to improve the exploration efficiency. However, in these methods, agents in different layers are typically trained independently and operate at fixed frequencies, which limits their performance and hampers their ability to respond to highly dynamic combat situations. In view of this, we present PJOH-TED2, a partial-joint-optimization-based hierarchical (PJOH) learning framework with a time-event dual-driven (TED2) mechanism, for one-on-one beyond-visual-range (BVR) air combat. Specifically, the PJOH learning framework embeds the partial joint optimization mechanism into hierarchical reinforcement learning (HRL), thus improving the exploration efficiency dramatically while enhancing the integration across hierarchical levels. Moreover, the TED2 mechanism combines the advantages of event-driven and time-driven methods, which promote the dynamic response speed of agents as well as avoid redundant actions. In addition, we evaluated this work through a series of games against the state-of-the-art (SOTA) methods in a high-fidelity air combat simulation environment. The results empirically demonstrate that the proposed approach outperforms four SOTA methods with a win rate of at least 71%. Finally, this approach achieved the 1st place in learning methods in the intelligent air game algorithm challenge (IAGAC) by the Chinese Institute of Command and Control among 43 teams.
- 11 April 2025
- Published by World Scientific Pub Co Pte Ltd in Guidance, Navigation and Control
- Vol. 05 (03) , 345-360
IOS-RAFS: Intelligent Opponent Selection Training Framework via Fuzzy Logic-Based Switching between “Rule-AI” Dual Strategy Libraries
In recent years, reinforcement learning-based aerial combat strategy design has shown significant advantages in decision-making efficiency and flexibility. However, improper opponent strategy selection during training may hinder the agent’s ability to handle diverse and complex scenarios, reducing the generalization and performance of its strategies. To address this, this paper proposes “IOS-RAFS”, a framework that dynamically adjusts opponent levels at different stages to provide a more diverse and adaptive environment. In the IOS-RAFS training framework, IOS refers to intelligent opponent selection, RA represents the “Rule-AI” dual strategy libraries, which are established to enhance the flexibility and diversity of opponent strategies, and FS represents a fuzzy logic-based method for switching between opponent strategy libraries, periodically selecting the strategy library for the next training phase based on training trends and model performance. Finally, the effectiveness of the proposed algorithm is validated through comparisons with the landmark methods in aerial combat, namely, the genetic fuzzy tree (GFT) algorithm and the expert-designed weapon engagement zones (WEZs) algorithm, demonstrating a significant improvement in rewards and achieving a win rate of at least 82.3%.
- 11 April 2025
- Published by World Scientific Pub Co Pte Ltd in Guidance, Navigation and Control
- Vol. 05 (03) , 345-360
IOS-RAFS: Intelligent Opponent Selection Training Framework via Fuzzy Logic-Based Switching between “Rule-AI” Dual Strategy Libraries
In recent years, reinforcement learning-based aerial combat strategy design has shown significant advantages in decision-making efficiency and flexibility. However, improper opponent strategy selection during training may hinder the agent’s ability to handle diverse and complex scenarios, reducing the generalization and performance of its strategies. To address this, this paper proposes “IOS-RAFS”, a framework that dynamically adjusts opponent levels at different stages to provide a more diverse and adaptive environment. In the IOS-RAFS training framework, IOS refers to intelligent opponent selection, RA represents the “Rule-AI” dual strategy libraries, which are established to enhance the flexibility and diversity of opponent strategies, and FS represents a fuzzy logic-based method for switching between opponent strategy libraries, periodically selecting the strategy library for the next training phase based on training trends and model performance. Finally, the effectiveness of the proposed algorithm is validated through comparisons with the landmark methods in aerial combat, namely, the genetic fuzzy tree (GFT) algorithm and the expert-designed weapon engagement zones (WEZs) algorithm, demonstrating a significant improvement in rewards and achieving a win rate of at least 82.3%.
- 16 February 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Emerging Topics in Computational Intelligence
- Vol. 8 (3) , 2182-2196
Autopilot Controller of Fixed-Wing Planes Based on Curriculum Reinforcement Learning Scheduled by Adaptive Learning Curve
In this paper, we present a novel curriculum reinforcement learning method that can automatically generate a high-performance autopilot controller for a 6-degree-of-freedom (6-DOF) aircraft with an unknown dynamic model, which is difficult to be handled using traditional control methods. In this method, a sigmoid-like learning curve is elegantly introduced to generate goals (the desired heading, altitude, and velocity) from easy to hard for autopilot. The shape of the learning curve can be intelligently adjusted to adapt to the training process of Proximal Policy Optimization (PPO). In addition, the conflict between multiple goals in autopilot training is solved by designing an adaptive reward function. Furthermore, the control inputs can avoid large oscillations by filtering the outputs from PPO with a first-order filter to ensure the smoothness. A series of simulation results show that the proposed method can not only observably improve the success rate and stability of training but also has superior performance in settling time and robustness compared with the traditional PID control and a state-of-the-art (SOTA) method. In the end, the applications of the controller, including the navigation task, pursuit-evasion, and dogfighting, are demonstrated to prove its feasibility to multiple tasks.
- 16 February 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Emerging Topics in Computational Intelligence
- Vol. 8 (3) , 2182-2196
Autopilot Controller of Fixed-Wing Planes Based on Curriculum Reinforcement Learning Scheduled by Adaptive Learning Curve
In this paper, we present a novel curriculum reinforcement learning method that can automatically generate a high-performance autopilot controller for a 6-degree-of-freedom (6-DOF) aircraft with an unknown dynamic model, which is difficult to be handled using traditional control methods. In this method, a sigmoid-like learning curve is elegantly introduced to generate goals (the desired heading, altitude, and velocity) from easy to hard for autopilot. The shape of the learning curve can be intelligently adjusted to adapt to the training process of Proximal Policy Optimization (PPO). In addition, the conflict between multiple goals in autopilot training is solved by designing an adaptive reward function. Furthermore, the control inputs can avoid large oscillations by filtering the outputs from PPO with a first-order filter to ensure the smoothness. A series of simulation results show that the proposed method can not only observably improve the success rate and stability of training but also has superior performance in settling time and robustness compared with the traditional PID control and a state-of-the-art (SOTA) method. In the end, the applications of the controller, including the navigation task, pursuit-evasion, and dogfighting, are demonstrated to prove its feasibility to multiple tasks.