Deep Reinforcement Learning-based Sum-Rate Maximization in Hybrid Beamforming Multi-User Massive MIMO Systems
Authors:
- 31 July 2024
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 601-606
Abstract
This work develops deep reinforcement learning (RL)-based techniques to perform power allocation in hybrid beamforming (HB) multi-user massive multiple-input multiple-output (MU-mMIMO) systems. We aim to maximize the achievable system sum-rate constrained to a total transmit power limit. The optimization problem is non-convex and solved in three main steps. First, a radio frequency (RF) beamformer is designed based on the slow time-varying angle of departure (AoD) information of users. Second, a baseband (BB) precoder is derived based on regularized zero-forcing technique using the lower-dimensional effective channel state information (CSI) seen from the BB stage. Afterward, the power allocation (PA) task is formulated in the reinforcement learning framework and it is solved by two main approaches: (i) single-agent deep Q-learning (SA-DQL), (ii) multi-agent deep Q-learning (MA-DQL). An RL agent learns to choose an action (i.e., PA block/value) that maximizes its expected cumulative reward (i.e., sum-rate) through interactions with the environment. It is shown that SA-DQL-based PA can closely approach the performance of exhaustive search in terms of achieved sum-rate and performs significantly better than the MA-DQL-based PA and equal PA scenarios.Keywords
This publication has 19 references indexed in Scilit:
- Dynamic Power Allocation for Cell-Free Massive MIMO: Deep Reinforcement Learning MethodsIEEE Access, 2021
- Taxonomy and Performance Evaluation of Hybrid Beamforming for 5G and Beyond SystemsIEEE Access, 2020
- A Comprehensive Survey on Millimeter Wave Communications for Fifth-Generation Wireless Networks: Feasibility and ChallengesIEEE Access, 2020
- 3D Angular-Based Hybrid Precoding and User Grouping for Uniform Rectangular Arrays in Massive MU-MIMO SystemsIEEE Access, 2020
- Q-Learning Algorithms: A Comprehensive Classification and ApplicationsIEEE Access, 2019
- Applications of Deep Reinforcement Learning in Communications and Networking: A SurveyIEEE Communications Surveys & Tutorials, 2019
- A Survey on Hybrid Beamforming Techniques in 5G: Architecture and System Model PerspectivesIEEE Communications Surveys & Tutorials, 2018
- Human-level control through deep reinforcement learningNature, 2015
- Dynamics of Boltzmannlearning in two-player two-action gamesPhysical Review E, 2012
- Independent reinforcement learners in cooperative Markov games: a survey regarding coordination problemsThe Knowledge Engineering Review, 2012