Reinforcement Learning of Beam Codebooks in Millimeter Wave and Terahertz MIMO Systems
- 9 November 2021
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Communications
- Vol. 70 (2) , 904-919
Abstract
Millimeter wave (mmWave) and terahertz MIMO systems rely on pre-defined beamforming codebooks for both initial access and data transmission. These pre-defined codebooks, however, are commonly not optimized for specific environments, user distributions, and/or possible hardware impairments. This leads to large codebook sizes with high beam training overhead which makes it hard for these systems to support highly mobile applications. To overcome these limitations, this paper develops a deep reinforcement learning framework that learns how to optimize the codebook beam patterns relying only on the receive power measurements. The developed model learns how to adapt the beam patterns based on the surrounding environment, user distribution, hardware impairments, and array geometry. Further, this approach does not require any knowledge about the channel, RF hardware, or user positions. To reduce the learning time, the proposed model designs a novel Wolpertinger -variant architecture that is capable of efficiently searching the large discrete action space. The proposed learning framework respects the RF hardware constraints such as the constant-modulus and quantized phase shifter constraints. Simulation results confirm the ability of the developed framework to learn near-optimal beam patterns for line-of-sight (LOS), non-LOS (NLOS), mixed LOS/NLOS scenarios and for arrays with hardware impairments without requiring any channel knowledge.Keywords
All Related Versions
Funding Information
- National Science Foundation (1923676)
This publication has 25 references indexed in Scilit:
- An Overview of Signal Processing Techniques for Millimeter Wave MIMO SystemsIEEE Journal of Selected Topics in Signal Processing, 2016
- Limited Feedback Hybrid Precoding for Multi-User Millimeter Wave SystemsIEEE Transactions on Wireless Communications, 2015
- Human-level control through deep reinforcement learningNature, 2015
- MIMO Precoding and Combining Solutions for Millimeter-Wave SystemsIEEE Communications Magazine, 2014
- Channel Estimation and Hybrid Precoding for Millimeter Wave Cellular SystemsIEEE Journal of Selected Topics in Signal Processing, 2014
- Efficient BackProp2012
- Equal gain transmission in multiple-input multiple-output wireless systemsIEEE Transactions on Communications, 2003
- Maximum ratio transmissionIEEE Transactions on Communications, 1999
- Reinforcement Learning: An IntroductionIEEE Transactions on Neural Networks, 1998
- The Hungarian method for the assignment problemNaval Research Logistics Quarterly, 1955