Abstract
Resource allocation and spectrum management are two of the major challenges in the massive scale deployment of Machine-to-Machine (M2M) communication and Internet of Things (IoT) networks. Existence of large number of devices per unit area leads to the problem of congestion, network overload and deterioration of the Signal to Noise Ratio (SNR). These problems can be eliminated through efficient resource allocation mechanisms which also play pivotal role in optimizing the throughput, delay, and power consumption of a network. Most of the existing resource allocation mechanisms are centralized and do not gracefully support the heterogeneous and dynamic IoT networks. Therefore, distributed and Machine Learning- (ML) based approaches are essential. Catering massive number of smart devices become challenging for the traditional distributed methods whereas ML-based approaches are currently scarce in the literature. To fill the voids, we propose a new distributed, block based Q-learning algorithm for slot scheduling of the smart devices / Machine Type Communication Devices (MTCDs) in the clustered IoT networks. We furthermore, propose various reward schemes for the evolution of Q-values in the proposed scheme and, discuss and evaluate its effect on the distributed model. The goal is to avoid inter-cluster and intra-cluster interference, and also to improve the Signal to Interference Ratio (SIR) by employing frequency diversity in a multi-channel system. Through extensive simulations, we analyze the effects of the distributed slotassignment with respect to varying SIR on the convergence rate and the convergence probability. Our theoretical analysis and simulations validate the effectiveness of our proposed method such that: (i) each MTCD is allocated with a suitable slot with acceptable SIR levels, (ii) IoT network can efficiently converge to a collision-free transmission causing least intracluster interferences, through each MTCD's learning ability in the process of the distributed slot allocation.

This publication has 32 references indexed in Scilit: