Reinforcement Learning for User Association and Handover in mmWave-Enabled Networks
- 7 June 2022
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Wireless Communications
- Vol. 21 (11) , 9712-9728
Abstract
Using a multi-armed bandit technique, we propose centralized and semi-distributed online algorithms for load balancing user association and handover in mmWave-enabled networks. Load balancing at all base stations (BSs) imposes explicit constraints that makes the actions of all user equipment (UEs) co-dependent, a challenging twist to reinforcement learning. We propose a central load balancer to guarantee load balancing at all BSs for every learning step. We consider two association vectors: one for leaning update, and one best-to-date for data transmission, allowing UEs to engage in best-result data transmission while effectively participating in a background learning process indefinitely. For dynamic networks, we introduce a measurement model capturing rapid channel variations and user mobility. To minimize handover rate, we also differentiate between handover costs for transmission and for learning, and introduce a learning handover cost decreasing with sojourn time. The proposed algorithms can be implemented online as they require no offline training and can effectively adapt to network dynamics. Numerical results show that the proposed algorithms exhibit fast learning convergence and outperform 3GPP handover by achieving an order of magnitude lower handover rate at a significantly higher network sum-rate, reaching within 94-97% of the near-optimal worst connection swapping benchmark algorithm.Keywords
Funding Information
- National Science Foundation under CNS (1908552)
This publication has 30 references indexed in Scilit:
- Multi-armed bandits in multi-agent networks2017
- A Learning Approach to Frequent Handover Mitigations in 3GPP Mobility Protocols2017
- Multi-armed bandits with application to 5G small cellsIEEE Wireless Communications, 2016
- Optimal User-Cell Association for Massive MIMO Wireless NetworksIEEE Transactions on Wireless Communications, 2015
- Millimeter Wave Cellular Networks: A MAC Layer PerspectiveIEEE Transactions on Communications, 2015
- Probabilistic Omnidirectional Path Loss Models for Millimeter-Wave Outdoor CommunicationsIEEE Wireless Communications Letters, 2015
- Joint Channel Selection and Power Control in Infrastructureless Wireless Networks: A Multiplayer Multiarmed Bandit FrameworkIEEE Transactions on Vehicular Technology, 2014
- 3D mmWave Channel Model Proposal2014
- User Association for Load Balancing in Heterogeneous Cellular NetworksIEEE Transactions on Wireless Communications, 2013
- Prediction, Learning, and Games2006