Chenxu Qian
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Research Subjects
Chenxu Qian has a total of 14 co-authors. Here are the top co-authors by number of shared publications:
- Xuebo Zhang
Institute of Robotics and Automatic Information Systems and Tianjin Key Laboratory of Intelligent Robotics, Nankai University, Tianjin, China
6 shared publications - Lun Li
Cyberspace Institute of Advanced Technology, Guangzhou University, Guangzhou, China
4 shared publications Lun Li
3 shared publications- Minghui Zhao
Nankai University
3 shared publications - Runhua Wang
Tianjin Key Laboratory of Intelligent Robotics, the College of Artificial Intelligence, and the Institute of Robotics and Automatic Information System, Nankai University, Tianjin, China
2 shared publications Xuebo Zhang
2 shared publications
Latest Publications
- Open Access
- 28 July 2026
- Published by Elsevier in SSRN Electronic Journal
Sparsity-Regularized Adaptive Coupling for Agile Flight Control of Fixed-Wing UAVs via Reinforcement Learning
Hybridizing reinforcement learning (RL) with imitation learning (IL) is attractive for fixed-wing UAV flight control, but sparse agile-maneuver rewards can make the transition from expert guidance to autonomous policy improvement unstable. Standard two-stage methods typically rely on rigid weight-copying warm starts to avoid early exploration failures, which may restrict subsequent policy adaptation when the expert is conservative. This paper presents Sparsity-Regularized Adaptive Coupling (SRAC) , a sparse-reward-aware imitation-regularized PPO framework for agile fixed-wing control. SRAC initializes the online RL policy from scratch rather than by copying expert weights, while using a frozen expert policy and value estimator as adaptive reference anchors during sparse-reward learning. A Reward Sparsity Estimator (RSE) measures the fraction of meaningful reward events in each training batch, and an Adaptive Imitation Anchor (AIA) adjusts the imitation penalty according to this feedback reliability. A State-Action Trust Mask (SATM) further gates the anchor by comparing the current critic with the frozen expert value estimator, reducing imitation pressure in regions where the learned policy appears more promising. Experiments in the JSBSim F-16 environment indicate that SRAC improves sparse-reward training stability and achieves faster transient maneuvers than the evaluated imitation-RL and flight-control baselines. The validation remains simulation-based, and the results should be interpreted together with the reported diagnostic sensitivity sweep, safety-envelope, and failure-mode analyses.
- 28 July 2026
- Published by Elsevier in SSRN Electronic Journal
Open AccessSparsity-Regularized Adaptive Coupling for Agile Flight Control of Fixed-Wing UAVs via Reinforcement Learning
Hybridizing reinforcement learning (RL) with imitation learning (IL) is attractive for fixed-wing UAV flight control, but sparse agile-maneuver rewards can make the transition from expert guidance to autonomous policy improvement unstable. Standard two-stage methods typically rely on rigid weight-copying warm starts to avoid early exploration failures, which may restrict subsequent policy adaptation when the expert is conservative. This paper presents Sparsity-Regularized Adaptive Coupling (SRAC) , a sparse-reward-aware imitation-regularized PPO framework for agile fixed-wing control. SRAC initializes the online RL policy from scratch rather than by copying expert weights, while using a frozen expert policy and value estimator as adaptive reference anchors during sparse-reward learning. A Reward Sparsity Estimator (RSE) measures the fraction of meaningful reward events in each training batch, and an Adaptive Imitation Anchor (AIA) adjusts the imitation penalty according to this feedback reliability. A State-Action Trust Mask (SATM) further gates the anchor by comparing the current critic with the frozen expert value estimator, reducing imitation pressure in regions where the learned policy appears more promising. Experiments in the JSBSim F-16 environment indicate that SRAC improves sparse-reward training stability and achieves faster transient maneuvers than the evaluated imitation-RL and flight-control baselines. The validation remains simulation-based, and the results should be interpreted together with the reported diagnostic sensitivity sweep, safety-envelope, and failure-mode analyses.
- 1 July 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Cybernetics
- Vol. 56 (7) , 3689-3702
SR-GRAT: Symmetric-Response Guided Reinforcement Learning With Adaptive Targeting for Agile Fixed-Wing UAV Control
This article presents a novel curriculum framework named symmetric-response guided reinforcement learning (RL) for autopilot control of fixed-wing aircraft, driven by an adaptive bidirectional learning curve and a dynamic target scheduling mechanism. Unlike traditional methods with static or overly smoothed learning progressions, the proposed method dynamically adjusts the learning curve's slope in both directions based on historical reward trends, allowing the learning intensity to increase or decrease as needed. This bidirectional adjustment ensures that the agent is neither overwhelmed by excessively difficult tasks nor stagnated by too-easy ones, leading to better stability and faster convergence. Furthermore, dynamic target generation within an episode from static target constraints enables both reward amplification and implicit enforcement of maneuver rate constraints, improving learning efficiency without manual reward shaping. Experiments on trajectory tracking tasks show that the proposed controller achieves faster convergence, reduced overshoot, and more accurate tracking under turbulence. Further tests on waypoint navigation and dynamic pursuit demonstrate its superiority over the baseline, achieving more precise and timely interception. These results highlight the robustness and applicability of the controller to complex aerial missions such as autonomous air combat.
- 1 July 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Cybernetics
- Vol. 56 (7) , 3689-3702
SR-GRAT: Symmetric-Response Guided Reinforcement Learning With Adaptive Targeting for Agile Fixed-Wing UAV Control
This article presents a novel curriculum framework named symmetric-response guided reinforcement learning (RL) for autopilot control of fixed-wing aircraft, driven by an adaptive bidirectional learning curve and a dynamic target scheduling mechanism. Unlike traditional methods with static or overly smoothed learning progressions, the proposed method dynamically adjusts the learning curve's slope in both directions based on historical reward trends, allowing the learning intensity to increase or decrease as needed. This bidirectional adjustment ensures that the agent is neither overwhelmed by excessively difficult tasks nor stagnated by too-easy ones, leading to better stability and faster convergence. Furthermore, dynamic target generation within an episode from static target constraints enables both reward amplification and implicit enforcement of maneuver rate constraints, improving learning efficiency without manual reward shaping. Experiments on trajectory tracking tasks show that the proposed controller achieves faster convergence, reduced overshoot, and more accurate tracking under turbulence. Further tests on waypoint navigation and dynamic pursuit demonstrate its superiority over the baseline, achieving more precise and timely interception. These results highlight the robustness and applicability of the controller to complex aerial missions such as autonomous air combat.
- Open Access
- 26 June 2026
- Published by Elsevier in SSRN Electronic Journal
Hybrid Offline-to-Online Kickstarting with Advantage-Weighted Regression for Agile Fixed-Wing UAV Flight Control
High-fidelity fixed-wing UAV flight control requires policies that can coordinate altitude, speed, and heading during rapid maneuvers while avoiding early exploration failures.This paper proposes Hybrid Offline-to-Online Kickstarting with Advantage-Weighted Regression (HOK-AWR), a single-stage hybrid training framework for agile fixed-wing flight control. HOK-AWR combines dynamic expert-student sampling, online relabeling of expert-action likelihoods under the current policy, and an advantage-weighted regularizer that down-weights conservative or low-advantage demonstrations as training progresses.To improve transient agility, we further design a sparse-then-dense hybrid reward that first encourages rapid maneuver completion and then stabilizes terminal tracking. Experiments in the JSBSim F-16 simulator show that HOK-AWR improves learning stability and maneuver execution over the evaluated imitation-augmented and DRL baselines, including an average 20% reduction in climbing settling time and improved 3D pursuit performance.
- 26 June 2026
- Published by Elsevier in SSRN Electronic Journal
Open AccessHybrid Offline-to-Online Kickstarting with Advantage-Weighted Regression for Agile Fixed-Wing UAV Flight Control
High-fidelity fixed-wing UAV flight control requires policies that can coordinate altitude, speed, and heading during rapid maneuvers while avoiding early exploration failures.This paper proposes Hybrid Offline-to-Online Kickstarting with Advantage-Weighted Regression (HOK-AWR), a single-stage hybrid training framework for agile fixed-wing flight control. HOK-AWR combines dynamic expert-student sampling, online relabeling of expert-action likelihoods under the current policy, and an advantage-weighted regularizer that down-weights conservative or low-advantage demonstrations as training progresses.To improve transient agility, we further design a sparse-then-dense hybrid reward that first encourages rapid maneuver completion and then stabilizes terminal tracking. Experiments in the JSBSim F-16 simulator show that HOK-AWR improves learning stability and maneuver execution over the evaluated imitation-augmented and DRL baselines, including an average 20% reduction in climbing settling time and improved 3D pursuit performance.
- 2 July 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Cybernetics
- Vol. 55 (9) , 4145-4157
A Partial Joint Optimization Algorithm for Autonomous Air Combat Based on Hierarchical Reinforcement Learning
Designing intelligent game strategies for autonomous air combat has suffered from the vast exploration space, lengthy decision-making process, and sparse rewards. Some existing approaches adopt the hierarchical framework to improve the exploration efficiency. However, in these methods, agents in different layers are typically trained independently and operate at fixed frequencies, which limits their performance and hampers their ability to respond to highly dynamic combat situations. In view of this, we present PJOH-TED2, a partial-joint-optimization-based hierarchical (PJOH) learning framework with a time-event dual-driven (TED2) mechanism, for one-on-one beyond-visual-range (BVR) air combat. Specifically, the PJOH learning framework embeds the partial joint optimization mechanism into hierarchical reinforcement learning (HRL), thus improving the exploration efficiency dramatically while enhancing the integration across hierarchical levels. Moreover, the TED2 mechanism combines the advantages of event-driven and time-driven methods, which promote the dynamic response speed of agents as well as avoid redundant actions. In addition, we evaluated this work through a series of games against the state-of-the-art (SOTA) methods in a high-fidelity air combat simulation environment. The results empirically demonstrate that the proposed approach outperforms four SOTA methods with a win rate of at least 71%. Finally, this approach achieved the 1st place in learning methods in the intelligent air game algorithm challenge (IAGAC) by the Chinese Institute of Command and Control among 43 teams.
- 2 July 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Cybernetics
- Vol. 55 (9) , 4145-4157
A Partial Joint Optimization Algorithm for Autonomous Air Combat Based on Hierarchical Reinforcement Learning
Designing intelligent game strategies for autonomous air combat has suffered from the vast exploration space, lengthy decision-making process, and sparse rewards. Some existing approaches adopt the hierarchical framework to improve the exploration efficiency. However, in these methods, agents in different layers are typically trained independently and operate at fixed frequencies, which limits their performance and hampers their ability to respond to highly dynamic combat situations. In view of this, we present PJOH-TED2, a partial-joint-optimization-based hierarchical (PJOH) learning framework with a time-event dual-driven (TED2) mechanism, for one-on-one beyond-visual-range (BVR) air combat. Specifically, the PJOH learning framework embeds the partial joint optimization mechanism into hierarchical reinforcement learning (HRL), thus improving the exploration efficiency dramatically while enhancing the integration across hierarchical levels. Moreover, the TED2 mechanism combines the advantages of event-driven and time-driven methods, which promote the dynamic response speed of agents as well as avoid redundant actions. In addition, we evaluated this work through a series of games against the state-of-the-art (SOTA) methods in a high-fidelity air combat simulation environment. The results empirically demonstrate that the proposed approach outperforms four SOTA methods with a win rate of at least 71%. Finally, this approach achieved the 1st place in learning methods in the intelligent air game algorithm challenge (IAGAC) by the Chinese Institute of Command and Control among 43 teams.
- 11 April 2025
- Published by World Scientific Pub Co Pte Ltd in Guidance, Navigation and Control
- Vol. 05 (03) , 345-360
IOS-RAFS: Intelligent Opponent Selection Training Framework via Fuzzy Logic-Based Switching between “Rule-AI” Dual Strategy Libraries
In recent years, reinforcement learning-based aerial combat strategy design has shown significant advantages in decision-making efficiency and flexibility. However, improper opponent strategy selection during training may hinder the agent’s ability to handle diverse and complex scenarios, reducing the generalization and performance of its strategies. To address this, this paper proposes “IOS-RAFS”, a framework that dynamically adjusts opponent levels at different stages to provide a more diverse and adaptive environment. In the IOS-RAFS training framework, IOS refers to intelligent opponent selection, RA represents the “Rule-AI” dual strategy libraries, which are established to enhance the flexibility and diversity of opponent strategies, and FS represents a fuzzy logic-based method for switching between opponent strategy libraries, periodically selecting the strategy library for the next training phase based on training trends and model performance. Finally, the effectiveness of the proposed algorithm is validated through comparisons with the landmark methods in aerial combat, namely, the genetic fuzzy tree (GFT) algorithm and the expert-designed weapon engagement zones (WEZs) algorithm, demonstrating a significant improvement in rewards and achieving a win rate of at least 82.3%.
- 11 April 2025
- Published by World Scientific Pub Co Pte Ltd in Guidance, Navigation and Control
- Vol. 05 (03) , 345-360
IOS-RAFS: Intelligent Opponent Selection Training Framework via Fuzzy Logic-Based Switching between “Rule-AI” Dual Strategy Libraries
In recent years, reinforcement learning-based aerial combat strategy design has shown significant advantages in decision-making efficiency and flexibility. However, improper opponent strategy selection during training may hinder the agent’s ability to handle diverse and complex scenarios, reducing the generalization and performance of its strategies. To address this, this paper proposes “IOS-RAFS”, a framework that dynamically adjusts opponent levels at different stages to provide a more diverse and adaptive environment. In the IOS-RAFS training framework, IOS refers to intelligent opponent selection, RA represents the “Rule-AI” dual strategy libraries, which are established to enhance the flexibility and diversity of opponent strategies, and FS represents a fuzzy logic-based method for switching between opponent strategy libraries, periodically selecting the strategy library for the next training phase based on training trends and model performance. Finally, the effectiveness of the proposed algorithm is validated through comparisons with the landmark methods in aerial combat, namely, the genetic fuzzy tree (GFT) algorithm and the expert-designed weapon engagement zones (WEZs) algorithm, demonstrating a significant improvement in rewards and achieving a win rate of at least 82.3%.
- 16 February 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Emerging Topics in Computational Intelligence
- Vol. 8 (3) , 2182-2196
Autopilot Controller of Fixed-Wing Planes Based on Curriculum Reinforcement Learning Scheduled by Adaptive Learning Curve
In this paper, we present a novel curriculum reinforcement learning method that can automatically generate a high-performance autopilot controller for a 6-degree-of-freedom (6-DOF) aircraft with an unknown dynamic model, which is difficult to be handled using traditional control methods. In this method, a sigmoid-like learning curve is elegantly introduced to generate goals (the desired heading, altitude, and velocity) from easy to hard for autopilot. The shape of the learning curve can be intelligently adjusted to adapt to the training process of Proximal Policy Optimization (PPO). In addition, the conflict between multiple goals in autopilot training is solved by designing an adaptive reward function. Furthermore, the control inputs can avoid large oscillations by filtering the outputs from PPO with a first-order filter to ensure the smoothness. A series of simulation results show that the proposed method can not only observably improve the success rate and stability of training but also has superior performance in settling time and robustness compared with the traditional PID control and a state-of-the-art (SOTA) method. In the end, the applications of the controller, including the navigation task, pursuit-evasion, and dogfighting, are demonstrated to prove its feasibility to multiple tasks.
- 16 February 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Emerging Topics in Computational Intelligence
- Vol. 8 (3) , 2182-2196
Autopilot Controller of Fixed-Wing Planes Based on Curriculum Reinforcement Learning Scheduled by Adaptive Learning Curve
In this paper, we present a novel curriculum reinforcement learning method that can automatically generate a high-performance autopilot controller for a 6-degree-of-freedom (6-DOF) aircraft with an unknown dynamic model, which is difficult to be handled using traditional control methods. In this method, a sigmoid-like learning curve is elegantly introduced to generate goals (the desired heading, altitude, and velocity) from easy to hard for autopilot. The shape of the learning curve can be intelligently adjusted to adapt to the training process of Proximal Policy Optimization (PPO). In addition, the conflict between multiple goals in autopilot training is solved by designing an adaptive reward function. Furthermore, the control inputs can avoid large oscillations by filtering the outputs from PPO with a first-order filter to ensure the smoothness. A series of simulation results show that the proposed method can not only observably improve the success rate and stability of training but also has superior performance in settling time and robustness compared with the traditional PID control and a state-of-the-art (SOTA) method. In the end, the applications of the controller, including the navigation task, pursuit-evasion, and dogfighting, are demonstrated to prove its feasibility to multiple tasks.
- 30 December 2023
- Published by Elsevier in Expert Systems with Applications
- Vol. 245
H3E: Learning air combat with a three-level hierarchical framework embedding expert knowledge
- 30 December 2023
- Published by Elsevier in Expert Systems with Applications
- Vol. 245
H3E: Learning air combat with a three-level hierarchical framework embedding expert knowledge
- 31 January 2023
- Published by Springer Nature in Neural Computing and Applications
- Vol. 35 (14) , 10239-10255
Basic flight maneuver generation of fixed-wing plane based on proximal policy optimization
- 31 January 2023
- Published by Springer Nature in Neural Computing and Applications
- Vol. 35 (14) , 10239-10255
Basic flight maneuver generation of fixed-wing plane based on proximal policy optimization