Self-Attention Proximal Policy Optimization for Beam Tracking in mmWave Communications
- 2 March 2026
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Journal on Selected Areas in Communications
- Vol. 44 (07338716) , 4552-4569
Abstract
Millimeter-wave (mmWave) communication systems are highly sensitive to user mobility and environmental changes, often suffering from rapid beam direction shifts and frequent link blockages. These dynamics pose significant challenges to maintaining reliable beam tracking. To enable accurate and robust beam tracking in dynamic environments, we propose a Self-Attention Proximal Policy Optimization (SAPPO) algorithm. The beam tracking task is modeled as a Partially Observable Markov Decision Process (POMDP), where the agent aims to maximize Received Signal Strength (RSS) using only current and previous signal observations, without relying on channel models or complex signal processing. By integrating a multi-head self-attention (MHSA) mechanism with a Gated Recurrent Unit (GRU), the policy network effectively captures temporal dependencies, enhancing the agent’s perception of user motion and channel state variations. To validate the approach, we construct three test scenarios with varying mobility and blockage complexity using the DeepMIMO dataset. Experimental results demonstrate that SAPPO consistently outperforms baseline methods such as DQN and PPO in terms of tracking accuracy, stability, and robustness. Notably, in challenging environments with frequent Line-Of-Sight (LOS)/Non-Line-Of-Sight (NLOS) transitions, SAPPO achieves a beam tracking success rate exceeding 95%, highlighting its strong adaptability and reliable performance under dynamic conditions.Keywords
Funding Information
- National Natural Science Foundation of China (62402499)
This publication has 68 references indexed in Scilit:
- Vision and LiDAR multi-modal fusion beam prediction method for millimeter-wave communication systemThe Journal of Supercomputing, 2025
- Efficient self-attention with smart pruning for sustainable large language modelsScientific Reports, 2025
- Robust Continuous-Time Beam Tracking with Liquid Neural Network2024
- Low-Rank Approximation for Sparse Attention in Multi-Modal LLMs2024
- Vision-driven metasurfaces for perception enhancementNature Communications, 2024
- Beam prediction and tracking mechanism with enhanced LSTM for mmWave aerial base stationWireless Networks, 2024
- Improved multistep ahead photovoltaic power prediction model based on LSTM and self-attention with weather forecast dataApplied Energy, 2024
- Machine Learning for Millimeter Wave and Terahertz Beam Management: A Survey and Open ChallengesIEEE Access, 2023
- Towards Real-World 6G Drone Communication: Position and Camera Aided Beam Prediction2022
- Adaptive particle filter based on Kullback–Leibler distance for underwater terrain aided navigation with multi‐beam sonarIET Radar, Sonar & Navigation, 2018