Deep Reinforcement Learning for mmWave Initial Beam Alignment
Preprint
Abstract
We investigate the applicability of deep reinforcement learning algorithms to the adaptive initial access beam alignment problem for mmWave communications using the state-of-the-art proximal policy optimization algorithm as an example. In comparison to recent unsupervised learning based approaches developed to tackle this problem, deep reinforcement learning has the potential to address a new and wider range of applications, since, in principle, no (differentiable) model of the channel and/or the whole system is required for training, and only agent-environment interactions are necessary to learn an algorithm (be it online or using a recorded dataset). We show that, although the chosen off-the-shelf deep reinforcement learning agent fails to perform well when trained on realistic problem sizes, introducing action space shaping in the form of beamforming modules vastly improves the performance, without sacrificing much generalizability. Using this add-on, the agent is able to deliver competitive performance to various state-of-the-art methods on simulated environments, even under realistic problem sizes. This demonstrates that through well-directed modification, deep reinforcement learning may have a chance to compete with other approaches in this area, opening up many straightforward extensions to other/similar scenarios.Keywords
All Related Versions
This publication has 8 references indexed in Scilit:
- Deep Reinforcement Learning Based Blind mmWave MIMO Beam AlignmentIEEE Transactions on Wireless Communications, 2022
- Active Sensing for Communications by LearningIEEE Journal on Selected Areas in Communications, 2022
- Reinforcement Learning of Beam Codebooks in Millimeter Wave and Terahertz MIMO SystemsIEEE Transactions on Communications, 2021
- Deep Active Learning Approach to Adaptive Beamforming for mmWave Initial AlignmentIEEE Journal on Selected Areas in Communications, 2021
- Fast mmwave Beam Alignment via Correlated Bandit LearningIEEE Transactions on Wireless Communications, 2019
- Active Learning and CSI Acquisition for mmWave Initial AlignmentIEEE Journal on Selected Areas in Communications, 2019
- Channel Estimation and Hybrid Precoding for Millimeter Wave Cellular SystemsIEEE Journal of Selected Topics in Signal Processing, 2014
- Spatially Sparse Precoding in Millimeter Wave MIMO SystemsIEEE Transactions on Wireless Communications, 2014