Dali Ding

With the wide application of uncrewed aerial vehicles (UAVs) in the military, multi-UAV cooperative air combat will become an important form of air combat. For the problem of multi-UAV swarm scale generalization, this paper proposes a biased random masked attention MAPPO (BRMA-MAPPO) algorithm, which achieves zero-shot scale generalization of multi-UAV swarms. Specifically, an entity observation encoder based on multi-head attention is introduced to address the variable number of multi-UAV swarms. Meanwhile, a biased random mask is proposed to enhance the performance at zero-shot scale generalization. The mask vector is generated by a mask vector generator network and applied to the attention matrix, masking non-essential entity information, thereby improving the generalization performance. The loss function of the mask vector generator network is constructed based on the information bottleneck theory, and the gradient of the mask network is derived. The simulation results show that the entity observation encoder proposed in this paper can help focus on key battlefield entity information and improve the training effect. The biased random mask proposed in this paper can mask non-important entity information, maintain focus on key entities when generalizing to larger scenarios, and improve generalization effects. Compared with the existing zero-shot generalization multi-agent reinforcement learning algorithms, BRMA-MAPPO can significantly improve the generalization performance. In the 8V8 and 10V10 zero-shot generalization scenarios, the method proposed in this paper outperformed state-of-the-art algorithms by 12% and 15% in terms of winning rate, respectively.

The unmanned aerial vehicle (UAV) pursuit–evasion game is the fundamental framework for promoting autonomous decision-making and collaborative control of multi-UAV systems. Faced with the limitations of current deep reinforcement learning methods in terms of transferability and generalization for scalable multi-fixed-wing UAV pursuit–evasion game scenarios, this paper proposes a hierarchical collaborative pursuit–evasion game framework based on target allocation and hierarchical reinforcement learning. The framework comprises three layers: target allocation layer, maneuver decision-making layer, and flight control layer. The target allocation layer employs a dynamic target assignment method based on a dynamic value adjustment mechanism, decomposing the multi-vs.-multi pursuit–evasion game into several one-vs.-one confrontations. The maneuver decision-making layer utilizes a maneuver decision-making method based on trajectory prediction and hierarchical reinforcement learning to generate adversarial maneuver commands. The flight control layer adopts a stable gradient-assisted reinforcement learning flight controller to ensure stable UAV flight. Comparisons with other algorithms across 3V3, 6V6, 9V9, and 12V12 scenarios demonstrate that the proposed method achieves high win rates in diverse game scales. The comparison results also demonstrate the advantages of the framework proposed in this paper in terms of training efficiency and large-scale scalability.

The aerial game environment is complex. To enhance mission success rates, UAVs must comprehensively consider threats from various directions and distances, as well as autonomous evasion maneuver decision-making methods for multiple UAV platforms, rather than solely focusing on threats from specific directions and distances or decision-making methods for fixed UAV platforms. Accordingly, this study proposes an autonomous missile evasion maneuver decision-making method for UAVs, suitable for multi-scenario and multi-platform transferable mission requirements. A three-dimensional UAV-missile pursuit-evasion model is established, along with state-space, hierarchical maneuver action space and reward function models for autonomous missile evasion. The auto-regressive multi-hybrid proximal policy optimization (ARMH-PPO) algorithm is proposed for this model, integrating autoregressive network structures and utilizing long short-term memory (LSTM) networks to extract temporal features. Drawing on exploration curriculum learning principles, temporal fusion of process and event reward functions is implemented to jointly guide the agent’s learning process through human experience and strategy exploration. Additionally, a proportion integration differentiation (PID) method is introduced to control the UAV’s maneuver execution, reducing the coupling between maneuver control quantities and the simulation object. Simulation experiments and result analysis demonstrate that the proposed algorithm ranks first in both average reward value and average evasion success rate metrics, with the average evasion success rate approximately 8% higher than the second-ranked algorithm. In the three initial scenarios where the missile is positioned laterally, head-on, and tail-behind the UAV, the UAV’s missile evasion success rates are 95%, 70%, and 85%, respectively. Multi-platform simulation results demonstrate that the decision model constructed in this paper exhibits a certain degree of multi-platform transferability.

The prediction of the tactical maneuver trajectory of target aircraft is an important component of unmanned aerial vehicle (UAV) autonomous air-to-air confrontation. In view of the shortcomings of low accuracy and poor real-time performance in the existing maneuver trajectory prediction methods, this paper establishes a hierarchical tactical maneuver trajectory prediction model to achieve maneuver trajectory prediction based on the prediction of target tactical maneuver intentions. First, extract the maneuver trajectory features and situation features from the above data to establish the classification rules of maneuver units. Second, a tactical maneuver unit prediction model is established using the deep echo-state network based on the auto-encoder with attention mechanism (DeepESN-AE-AM) to predict 21 basic maneuver units. Then, for the above-mentioned 21 basic maneuver units, establish a maneuver trajectory prediction model using the gate recurrent unit based on triangle search optimization with attention mechanism (TSO-GRU-AM). Finally, by integrating the above two prediction models, a hierarchical strategy is adopted to establish a tactical maneuver trajectory prediction model. A section of the confrontation trajectory is selected from the air-to-air confrontation simulation data for prediction, and the results show that the trajectory prediction error of the combination of DeepESN-AE-AM and TSO-GRU-AM is small and meets the accuracy requirements. The simulation results of three air-to-air confrontation scenarios show that the proposed trajectory prediction method helps to assist UAV in accurately judging the confrontational situation and selecting high-quality maneuver strategies.

In recent years, autonomous maneuver decision-making has emerged as a key technology in autonomous air combat confrontation, garnering widespread attention. A method combining the modified marine predator algorithm (MMPA) and fuzzy inference is proposed to solve the autonomous maneuver decision-making problem of an unmanned combat aerial vehicle (UCAV). By incorporating the missile attack strategy into the process of calculating the maneuver strategy, the air combat decision-making capability of the UCAV is enhanced. First, the weight coefficients determined by the fuzzy inference method are combined with air combat superiority functions that consider the current missile attack zone and then the objective function is obtained, which is to be optimized at the current moment. Second, the MMPA is used to solve the objective function to obtain the missile attack maneuver strategy and the maneuver strategy for defending against missile attacks. A comparative analysis with other classical intelligent optimization algorithms highlights the advantages of the proposed method. Furthermore, the air combat confrontation simulation experiments are conducted under six different initial scenarios, namely, neutral, offensive, oppositional, defensive, parallel, and head-on. The simulation results show that the integrated maneuver and missile attack decision-making capabilities of the UCAV are improved using the proposed autonomous maneuver decision-making method.

Targeting the autonomous decision-making problem of unmanned combat aerial vehicles (UCAVs) in a two-versus-one (2v1) within-visual-range (WVR) air combat scenario, this paper proposes a maneuver decision-making method based on tactical coordination. First, a coordinated situation assessment model is designed, which subdivides the air combat situation into optimization-driven and tactical coordinated situations. The former combines missile attack zone calculation and trajectory prediction to optimize the control quantity of a single aircraft, while the latter uses fuzzy logic to analyze the overall situation of the three aircraft to drive tactical selection. Second, a decision-making model based on a hierarchical expert system is constructed, establishing a hierarchical decision-making framework with a UCAV-coordinated combat knowledge base. The coordinated situation assessment results are used to match corresponding tactics and maneuver control quantities. Finally, an improved particle swarm optimization algorithm (I-PSO) is proposed, which enhances the optimization ability and real-time performance through the design of local social factor iterative components and adaptive adjustment of inertia weights. Air combat simulations in four different scenarios verify the effectiveness and superiority of the proposed decision-making method. The results show that the method can achieve autonomous decision making in dynamic air combat. Compared with decision-making methods based on optimization algorithms and differential games, the win rate is increased by about 17% and 18%, respectively, and the single-step decision-making time is less than 0.02 s, demonstrating high real-time performance and win rate. This research provides new ideas and methods for the autonomous decision making of UCAVs in complex air combat scenarios.

One-to-one within-visual-range air combat of unmanned combat aerial vehicles (UCAVs) requires fast, continuous, and accurate decision-making to achieve air combat victory. In order to solve the current problems of insufficient real-time performance of traditional intelligent optimization algorithms for solving decision-making problems and the mismatch between the planning trajectory and the actual flight trajectory caused by the difference between the decision-making model and the actual aircraft model, this paper proposes a hierarchical on-line air combat maneuvering decision-making and control framework. Considering the real-time constraints, the maneuver decision problem is transformed into an expensive optimization problem at the decision planning layer. The surrogate-assisted differential evolution algorithm is proposed on the basis of the original differential evolution algorithm, and the planning trajectory is obtained through the 5 degrees of freedom (DOF) model. In the control execution layer, the planning trajectory is tracked through the nonlinear dynamic inverse tracking control method to realize the high-precision control of the 6DOF model. The simulation is carried out under four different initial situation scenarios, including head-on neutral, dominant, parallel neutral, and disadvantaged situations. The Monte Carlo simulation results show that the Surrogate-assisted differential evolution algorithm (SADE) can achieve a win rate of over 53% in all four initial scenarios. The proposed maneuver decision and control framework in this article achieves smooth flight trajectories and stable aircraft control, with each decision average taking 0.08 s, effectively solving the real-time problem of intelligent optimization algorithms in maneuver decision problems.

In this paper, an autonomous UCAV air combat maneuvering decision method based on LSHADE-TSO optimization in a model predictive control framework is proposed, along with enemy trajectory prediction. First, a sliding window recursive prediction method for multi-step enemy trajectory prediction using a Bi-LSTM network is proposed. Second, Model Predictive Control (MPC) theory is introduced, and when combined with enemy trajectory prediction, a UCAV maneuver decision model based on the MPC framework is proposed. The LSHADE-TSO algorithm is proposed by combining the LSHADE and TSO algorithms, which overcomes the problem of traditional sequential quadratic programming falling into local optimum when solving complex nonlinear models. The LSHADE-TSO-MPC air combat maneuver decision method is then proposed, which combines the LSHADE-TSO algorithm with the MPC framework and employs the LSHADE-TSO algorithm as the optimal control sequence solver. To validate the effectiveness of the maneuvering decision method proposed in this paper, it is tested against the test maneuver and the LSHADE-TSO decision algorithm, respectively, and the experimental results show that the maneuvering decision method proposed in this paper can beat the opponent and win the air combat using the same weapons and flight platform. Finally, to demonstrate that LSHADE-TSO can better exploit the decision-making ability of the MPC model, LSHADE-TSO is compared to various optimization algorithms based on the MPC model, and the results show that LSHADE-TSO-MPC can not only help obtain air combat victory faster but also demonstrates better decision-making ability.

In this paper, we propose IJADE-TSO, a novel hybrid algorithm in which an improved adaptive differential evolution with optional external archive (IJADE) has been combined with the tuna swarm optimization (TSO). The proposed algorithm incorporates the spiral foraging search and parabolic foraging search of TSO into the mutation strategy in IJADE to improve the exploration ability and population diversity. Additionally, to enhance the convergence efficiency, crossover factor (CR) ranking, CR repairing, top α r₁ selection, and population linear reduction strategies have been included in the algorithm. To evaluate the superiority of the proposed algorithm, IJADE-TSO has been benchmarked with its state-of-the-art counterparts using the CEC 2014 test set. Finally, to check the validity of IJADE-TSO, we apply it to photovoltaic (PV) parameter identification and compare its performance with those of other recently developed well-known algorithms. The statistical results reveal that IJADE-TSO outperforms the other compared algorithms.