Dynamic spectrum access with deep Q-learning in densely occupied and partially observable environments
- 1 January 2021
- journal article
- research article
- Published by Centre for Evaluation in Education and Science (CEON/CEES) in Telfor Journal
- Vol. 13 (1) , 1-6
Abstract
In this paper, we propose a new Dynamic Spectrum Access (DSA) method for multi-channel wireless networks. We assume that DSA nodes do not have prior knowledge of the system dynamics, and have only partial observability of the channels. Thus, the problem is formulated as a Partially Observable Markov Decision Process (POMDP) with exponential time complexity. We have developed a novel Deep Reinforcement Learning (DRL) based DSA method which combines a double deep Q-learning architecture with a recurrent neural network and takes advantage of a prioritized experience buffer. The simulation analysis shows that the proposed method accurately predicts a channel state based on the fixed-length history of partial observations. Compared with other DRL methods for DSA, the proposed solution can find a near-optimal policy in a smaller number of iterations and suits a wider range of communication environments, including dynamic ones, where channel occupancy pattern changes over time. The performance improvement increases with the number of channels and with a channel state transition uncertainty. To boost the performance of the algorithm in densely occupied environments, multiple DRL exploration strategies are examined and evaluation results are presented in the paper.Keywords
This publication has 16 references indexed in Scilit:
- Deep Reinforcement Learning for Dynamic Multichannel Access in Wireless NetworksIEEE Transactions on Cognitive Communications and Networking, 2018
- A decentralized platform for heterogeneous IoT networks management2018
- An architecture for QoS-aware service deployment in software-defined IoT networks2017
- The IoT Architectural Framework, Design Issues and Application DomainsWireless Personal Communications, 2016
- LSTM: A Search Space OdysseyIEEE Transactions on Neural Networks and Learning Systems, 2016
- SDN Based Mobile Networks: Concepts and BenefitsWireless Personal Communications, 2014
- Multiband Spectrum Access: Great Promises for Future Cognitive Radio NetworksProceedings of the IEEE, 2014
- Self-Regulating Action Exploration in Reinforcement LearningProcedia Computer Science, 2012
- Indexability of Restless Bandit Problems and Optimality of Whittle Index for Dynamic Multichannel AccessIEEE Transactions on Information Theory, 2010
- Optimality of Myopic Sensing in Multichannel Opportunistic AccessIEEE Transactions on Information Theory, 2009