Yujie Cui

Deep reinforcement learning (DRL) is extensively applied in autonomous unmanned aerial vehicle (UAV) control yet faces critical challenges regarding adaptability and generalization in dynamic environments. To address these limitations, this paper proposes the Meta Gated Transformer-XL Soft Actor-Critic (Meta-GSAC) algorithm. This framework integrates a Gated Transformer-XL module to capture long-term temporal dependencies from multimodal inputs and incorporates the Reptile algorithm to facilitate multi-task meta-learning. Experimental results demonstrate that Meta-GSAC significantly outperforms standard baselines. Notably, it achieves optimal policy convergence with approximately 50% fewer training epochs while effectively eliminating the high-frequency control oscillations observed in the GSAC baseline. Moreover, the proposed method exhibits superior few-shot adaptation capabilities, enabling the UAV to rapidly adapt to novel task scenarios with minimal gradient updates.

Research on UAV (unmanned aerial vehicle) path planning and obstacle avoidance control based on DRL (deep reinforcement learning) still faces limitations, as previous studies primarily utilized current perceptual inputs while neglecting the continuity of flight processes, resulting in low early-stage learning efficiency. To address these issues, this paper integrates DRL with the Transformer architecture to propose the GTrXL-SAC (gated Transformer-XL soft actor critic) algorithm. The algorithm performs positional embedding on multimodal data combining visual and sensor information. Leveraging the self-attention mechanism of GTrXL, it effectively focuses on different segments of multimodal data for encoding while capturing sequential relationships, significantly improving obstacle recognition accuracy and enhancing both learning efficiency and sample efficiency. Additionally, the algorithm capitalizes on GTrXL’s memory characteristics to generate current drone control decisions through the combined analysis of historical experiences and present states, effectively mitigating long-term dependency issues. Experimental results in the AirSim drone simulation environment demonstrate that compared to PPO and SAC algorithms, GTrXL-SAC achieves more precise policy exploration and optimization, enabling superior control of drone velocity and attitude for stabilized flight while accelerating convergence speed by nearly 20%.