Deep Multi-User Reinforcement Learning for Dynamic Spectrum Access in Multichannel Wireless Networks
Authors:
- Oshri NaparstekONOshri Naparstek
- Kobi CohenKCKobi Cohen
- 1 December 2017
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 1-7
Abstract
We consider the problem of dynamic spectrum access for network utility maximization in multichannel wireless networks. The shared bandwidth is divided into K orthogonal channels, and the users access the spectrum using a random access protocol. In the beginning of each time slot, each user selects a channel and transmits a packet with a certain attempt probability. After each time slot, each user that has transmitted a packet receives a local observation indicating whether its packet was successfully delivered or not (i.e., ACK signal). The objective is to find a multi-user strategy that maximizes a certain network utility in a distributed manner without online coordination or message exchanges between users. Obtaining an optimal solution for the spectrum access problem is computationally expensive in general due to the large state space and partial observability of the states. To tackle this problem, we develop a distributed dynamic spectrum access algorithm based on deep multi-user reinforcement leaning. Specifically, at each time slot, each user maps its current state to spectrum access actions based on a trained deep-Q network used to maximize the objective function. Experimental results have demonstrated that users are capable to learn good policies that achieve strong performance in this challenging partially observable setting only from their ACK signals, without online coordination, message exchanges between users, or carrier sensing.Keywords
All Related Versions
This publication has 23 references indexed in Scilit:
- Matching theory for future wireless networks: fundamentals and applicationsIEEE Communications Magazine, 2015
- Human-level control through deep reinforcement learningNature, 2015
- Game Theoretic Aspects of the Multi-Channel ALOHA Protocol in Cognitive Radio NetworksIEEE Journal on Selected Areas in Communications, 2013
- Price of Anarchy for Congestion Games in Cognitive Radio NetworksIEEE Transactions on Wireless Communications, 2012
- Approximately optimal adaptive learning in opportunistic spectrum access2012
- Indexability of Restless Bandit Problems and Optimality of Whittle Index for Dynamic Multichannel AccessIEEE Transactions on Information Theory, 2010
- Multiagent -Learning for Aloha-Like Spectrum Access in Cognitive Radio SystemsEURASIP Journal on Wireless Communications and Networking, 2010
- Decentralized cognitive MAC for opportunistic spectrum access in ad hoc networks: A POMDP frameworkIEEE Journal on Selected Areas in Communications, 2007
- Achieving Proportional Fairness Using Local Information in Aloha NetworksIEEE Transactions on Automatic Control, 2004
- Q-learningMachine Learning, 1992