Transfer Reinforcement Learning for Dynamic Spectrum Environment
- 5 July 2023
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Wireless Communications
- Vol. 23 (2) , 1447-1458
Abstract
Reinforcement learning (RL) has proven to be an effective approach for achieving intelligence in Cognitive Radio (CR). Through interactions with the environment, RL enables a CR to optimize in an efficient and flexible manner. The vast majority of studies, however, are carried out in a spectrum environment with prefixed user access rules, typically with a constant transition probability and reward distribution. In fact, in a real-world spectrum environment, changes in access rules are common, which has a significant impact on the effectiveness of RL, while few studies have been conducted on this topic. This paper demonstrates how changes in primary user’s (PU) access rules affect RL strategies. To improve the secondary user’s (SU) performance for the dynamic spectrum environment, a transfer Deep Q-Network (DQN) is proposed, this method screens out knowledge from historical experience while avoiding interference from irrelevant information with an experience playback mechanism. Experiments show that this method outperforms traditional RL methods in terms of conflict rate, spectrum utilization rate, and convergence rate in the dynamic spectrum. Given the scarcity of studies on this topic, this study is expected to serve as a benchmark for the future research.Keywords
Funding Information
- National Natural Science Foundation of China (61903066)
- China Postdoctoral Science Foundation (2021M690560)
- Shenzhen Institute of Artificial Intelligence and Robotics for Society
This publication has 43 references indexed in Scilit:
- Distributed learning algorithm with synchronized epochs for dynamic spectrum access in unknown environment using multi-user restless multi-armed banditJournal of King Saud University Computer and Information Sciences, 2022
- Energy-efficient cooperative spectrum sensing over imperfect reporting channels with collision constraintPhysical Communication, 2021
- A segment-graph algorithm for two-objective wireless spectrum allocation in cognitive networksComputer Communications, 2021
- Intelligent spectrum management based on reinforcement learning schemes in cooperative cognitive radio networksPhysical Communication, 2020
- Scarce-resource capacity sharing in cognitive radio environments: a new game theoretical modelTelecommunication Systems, 2017
- Human-level control through deep reinforcement learningNature, 2015
- Sequential sensing based spectrum handoff in cognitive radio networks with multiple usersComputer Networks, 2014
- Cognitive radio network architectures2010
- Cognitive radio: making software radios more personalIEEE Wireless Communications, 1999
- Self-improving reactive agents based on reinforcement learning, planning and teachingMachine Learning, 1992