Imperfect CSI Based Intelligent Dynamic Spectrum Management Using Cooperative Reinforcement Learning Framework in Cognitive Radio Networks
- 24 September 2020
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Mobile Computing
- Vol. 21 (5) , 1672-1683
Abstract
The rapid development of wireless traffic pushed the wireless community to research different solutions towards the efficient utilization of the available radio spectrum. However, a recent study shows that most of the dynamically allocated spectrum bands, experience significant under-utilization as Cognitive Radio (CR) technology still lacks intelligence. An intelligence in CRs can be incorporated with machine learning algorithms. Further, the perfect Channel State Information (CSI) is hardly obtained and CSI imperfections play a crucial role in Spectrum Management. Thus, for efficient utilization of available spectrum, a decentralized Multi-Agent Reinforcement Learning based resource allocation scheme has been proposed. A robust scheme is proposed which integrates machine learning and CR technology into a sophisticated Multi-Agent System (MAS). Moreover, assisted with cloud computing provides huge amount of storage space, reduces operating expenditures, and provides wider flexibility of cooperation. Hence, to foster the performance of the proposed scheme, a cooperative framework in MAS is introduced which enhances the performance of the proposed scheme in terms of network capacity, outage probability, and convergence speed. Numerical results verify the effectiveness of proposed scheme and show the non-negligible impact of imperfect CSI, thus highlighting the importance of robust designs that maintains users' QoS in practical wireless networks.Keywords
Funding Information
- Department of Science and Technology, New Delhi, Government of India (SR/WOS-A/ET-52/2017)
- Anusandhan National Research Foundation (EEQ/2017/000592)
This publication has 22 references indexed in Scilit:
- A Survey on Applications of Model-Free Strategy Learning in Cognitive Wireless NetworksIEEE Communications Surveys & Tutorials, 2016
- Spectrum handoff in cognitive radio networks: A classification and comprehensive surveyJournal of Network and Computer Applications, 2016
- Distributed Heuristically Accelerated Q-Learning for Robust Cognitive Spectrum Management in LTE Cellular SystemsIEEE Transactions on Mobile Computing, 2015
- Imperfect CSI-Based Joint Resource Allocation in Multirelay OFDMA NetworksIEEE Transactions on Vehicular Technology, 2014
- Performance of distributed multi-agent multi-state reinforcement spectrum management using different exploration schemesExpert Systems with Applications, 2013
- A Survey on Machine-Learning Techniques in Cognitive RadiosIEEE Communications Surveys & Tutorials, 2012
- Two-stage reinforcement-learning-based cognitive radio with exploration controlIET Communications, 2011
- A Unified Analysis of Value-Function-Based Reinforcement-Learning AlgorithmsNeural Computation, 1999
- Reinforcement Learning: An IntroductionIEEE Transactions on Neural Networks, 1998
- Q-learningMachine Learning, 1992