Reinforcement Learning With Conformal Symplectic Optimization for Aerial RIS-Aided Secure Communication
- 11 March 2026
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Wireless Communications
- Vol. 25 (15361276) , 13560-13574
Abstract
This article investigates a secure aerial reconfigurable intelligent surface (A-RIS) communication system, where user mobility, imperfect channel state information (CSI), and RIS phase errors induced by uncrewed aerial vehicle (UAV) jitter significantly degrade performance. To address these challenges, we formulate a joint optimization problem for UAV trajectory, base station (BS) we propose abeamforming, and A-RIS beamforming to maximize the minimum secrecy energy efficiency (SEE), subject to constraints on user secrecy rates and UAV energy efficiency. To solve this highly non-convex problem, we propose a novel reinforcement learning framework termed IA-CSORL based on the twin-twin-delayed deep deterministic policy gradient (TTD3) architecture, which incorporates two novel modules. Specifically, we develop the phase-aware relativistic adaptive descent (PRAD) algorithm is proposed, which embeds the learning process into a conformal Hamiltonian system. By integrating gradient-based phase error correction and adaptive momentum adjustment, PRAD effectively counteracts phase noise and stabilizes training. Furthermore, we design an environment-state interactive attention (ESIA) mechanism to dynamically fuse UAV positioning and environmental features, enhancing state representation and deployment accuracy. Numerical results demonstrate that IA-CSORL significantly outperforms existing RL baselines in terms of both robustness and convergence performance. Moreover, IA-CSORL achieves superior beamforming accuracy under phase errors and CSI imperfections and provides a better trade-off between sum secrecy rate (SSR) and SEE, with performance gains becoming more significant as the number of RIS elements increases.Keywords
Funding Information
- National Natural Science Foundation of China (62501184)
- National Natural Science Foundation of China (U23A20271)
- National Natural Science Foundation of China (62501184)
- European Union through the Horizon Europe Project COVER (101086228)
- Horizon Europe Project UNITE (101129618)
- Horizon Europe Project INSTINCT (101139161)
- Horizon Europe project TWIN6G (101182794)
- Agence Nationale de la Recherche through the France 2030 Project ANR-PEPR Networks of the Future (YACARI 22-PEFT-0005)
- CHIST-ERA Project PASSIONATE (CHIST-ERA-22-WAI-04)
- CHIST-ERA Project PASSIONATE (ANR-23-CHR4-0003-01)
- Engineering and Physical Sciences Research Council
- UK Research and Innovation, in part by the UK Department of Science, Innovation and Technology (EP/X040518/1)
- UK Research and Innovation, in part by the UK Department of Science, Innovation and Technology (EP/Y037421/1)
- HASC Telecom Hub (EP/X040569/1)
This publication has 51 references indexed in Scilit:
- Overview of AI and communication for 6G network: fundamentals, challenges, and future research opportunitiesScience China Information Sciences, 2025
- The small-drone revolution is coming — scientists need to ensure it will be safeNature, 2024
- Terahertz Communications and Sensing for 6G and Beyond: A Comprehensive ReviewIEEE Communications Surveys & Tutorials, 2024
- Collision probability estimation for small unmanned aircraft systemsReliability Engineering & System Safety, 2021
- Reconfigurable Intelligent Surfaces: Principles and OpportunitiesIEEE Communications Surveys & Tutorials, 2021
- Conformal symplectic and relativistic optimizationJournal of Statistical Mechanics: Theory and Experiment, 2020
- Joint Beamforming and Trajectory Optimization for Intelligent Reflecting Surfaces-Assisted UAV CommunicationsIEEE Access, 2020
- Applications of Deep Reinforcement Learning in Communications and Networking: A SurveyIEEE Communications Surveys & Tutorials, 2019
- Reinforcement learning for control: Performance, stability, and deep approximatorsAnnual Reviews in Control, 2018
- Conformal Hamiltonian systemsJournal of Geometry and Physics, 2001