A New Block-Based Reinforcement Learning Approach for Distributed Resource Allocation in Clustered IoT Networks
- 10 January 2020
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Vehicular Technology
- Vol. 69 (3) , 2891-2904
Abstract
Resource allocation and spectrum management are two of the major challenges in the massive scale deployment of Machine-to-Machine (M2M) communication and Internet of Things (IoT) networks. Existence of large number of devices per unit area leads to the problem of congestion, network overload and deterioration of the Signal to Noise Ratio (SNR). These problems can be eliminated through efficient resource allocation mechanisms which also play pivotal role in optimizing the throughput, delay, and power consumption of a network. Most of the existing resource allocation mechanisms are centralized and do not gracefully support the heterogeneous and dynamic IoT networks. Therefore, distributed and Machine Learning- (ML) based approaches are essential. Catering massive number of smart devices become challenging for the traditional distributed methods whereas ML-based approaches are currently scarce in the literature. To fill the voids, we propose a new distributed, block based Q-learning algorithm for slot scheduling of the smart devices / Machine Type Communication Devices (MTCDs) in the clustered IoT networks. We furthermore, propose various reward schemes for the evolution of Q-values in the proposed scheme and, discuss and evaluate its effect on the distributed model. The goal is to avoid inter-cluster and intra-cluster interference, and also to improve the Signal to Interference Ratio (SIR) by employing frequency diversity in a multi-channel system. Through extensive simulations, we analyze the effects of the distributed slotassignment with respect to varying SIR on the convergence rate and the convergence probability. Our theoretical analysis and simulations validate the effectiveness of our proposed method such that: (i) each MTCD is allocated with a suitable slot with acceptable SIR levels, (ii) IoT network can efficiently converge to a collision-free transmission causing least intracluster interferences, through each MTCD's learning ability in the process of the distributed slot allocation.Keywords
This publication has 32 references indexed in Scilit:
- Base Station Selection in M2M Communication Using Q-Learning Algorithm in LTE-A Networks2015
- The challenges of M2M massive access in wireless cellular networksDigital Communications and Networks, 2015
- Energy-Efficiency Oriented Traffic Offloading in Wireless Networks: A Brief Survey and a Learning Approach for Heterogeneous Cellular NetworksIEEE Journal on Selected Areas in Communications, 2015
- Channel Selection for Network-Assisted D2D Communication via No-Regret Bandit Learning With Calibrated ForecastingIEEE Transactions on Wireless Communications, 2014
- K-means based clustering approach for data aggregation in periodic sensor networks2014
- Medium access control techniques in M2M communication: survey and critical reviewTransactions on Emerging Telecommunications Technologies, 2014
- An hybrid clustering algorithm for optimal clusters in Wireless sensor networks2014
- Resource allocation for uplink M2M communication: A game theory approach2013
- Reinforcement learning approach to dynamic activation of base station resources in wireless networks2013
- A bird's eye view on reinforcement learning approaches for power management in WSNs2013