Transmission Control in NB-IoT With Model-Based Reinforcement Learning
Open Access
- 9 June 2023
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 11, 57991-58005
Abstract
In Narrowband Internet of Things (NB-IoT), the control of uplink transmissions is a complex task involving device scheduling, resource allocation in the carrier, and the configuration of link-adaptation parameters. Existing heuristic proposals partially address the problem, but reinforcement learning (RL) seems to be the most effective approach a priori, given its success in similar control problems. However, the low sample efficiency of conventional (model-free) RL algorithms is an important limitation for their deployment in real systems. During their initial learning stages, RL agents need to explore the policy space selecting actions that are, in general, highly ineffective. In an NBIoT access network this implies a disproportionate increase in transmission delays. In this paper, we make two contributions to enable the adoption of RL in NBIoT: first, we present a multi-agent architecture based on the principle of task division. Second, we propose a new model-based RL algorithm for link adaptation characterized by its high sample efficiency. The combination of these two strategies results in an algorithm that, during the learning phase, is able to maintain the transmission delay in the order of hundreds of milliseconds, whereas model-free RL algorithms cause delays of up to several seconds. This allows our approach to be deployed, without prior training, in an operating NBIoT network and learn to control it efficiently without degrading its performance.Keywords
This publication has 29 references indexed in Scilit:
- Design of a UE-specific uplink scheduler for narrowband Internet-of-Things (NB-IoT) systems2018
- Narrowband IoT Data Transmission Procedures for Massive Machine-Type CommunicationsIEEE Network, 2017
- Cooperative Multi-agent Control Using Deep Reinforcement Learning2017
- Prediction-Based Energy Saving Mechanism in 3GPP NB-IoT NetworksSensors, 2017
- Online Optimization of Interference Coordination Parameters in Small Cell NetworksIEEE Transactions on Wireless Communications, 2017
- A Primer on 3GPP Narrowband Internet of ThingsIEEE Communications Magazine, 2017
- Uplink Scheduling and Link Adaptation for Narrowband Internet of Things SystemsIEEE Access, 2017
- Human-level control through deep reinforcement learningNature, 2015
- Building Controllers for TetrisICGA Journal: The Journal of the Computer Games Community, 2009
- Mining time-changing data streams2001