A Survey on Applications of Model-Free Strategy Learning in Cognitive Wireless Networks
Authors:
Open Access
- 9 March 2016
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Communications Surveys & Tutorials
- Vol. 18 (3) , 1717-1757
Abstract
The framework of cognitive wireless networks is expected to endow the wireless devices with the cognition-intelligence ability with which they can efficiently learn and respond to the dynamic wireless environment. In many practical scenarios, the complexity of network dynamics makes it difficult to determine the network evolution model in advance. Thus, the wireless decision-making entities may face a black-box network control problem and the model-based network management mechanisms will be no longer applicable. In contrast, model-free learning enables the decision-making entities to adapt their behaviors based on the reinforcement from their interaction with the environment and (implicitly) build their understanding of the system from scratch through trial-and-error. Such characteristics are highly in accordance with the requirement of cognition-based intelligence for devices in cognitive wireless networks. Therefore, model-free learning has been considered as one key implementation approach to adaptive, self-organized network control in cognitive wireless networks. In this paper, we provide a comprehensive survey on the applications of the state-of-the-art model-free learning mechanisms in cognitive wireless networks. According to the system models on which those applications are based, a systematic overview of the learning algorithms in the domains of single-agent system, multiagent systems, and multiplayer games is provided. The applications of model-free learning to various problems in cognitive wireless networks are discussed with the focus on how the learning mechanisms help to provide the solutions to these problems and improve the network performance over the model-based, non-adaptive methods. Finally, a broad spectrum of challenges and open issues is discussed to offer a guideline for the future research directions.Keywords
All Related Versions
Funding Information
- National Science Foundation (CCF-1331788, ECCS-1547201, CCF-1456921, CNS-1443917, ECCS-1405121, NSFC61428101)
- Korean government (2014R1A5A1011478)
- Singapore MOE Tier 1 (RG18/13, RG33/12)
- MOE Tier 2 (MOE2014-T2-2-015 ARC 4/15)
This publication has 176 references indexed in Scilit:
- A stochastic game framework for joint frequency and power allocation in dynamic decentralized cognitive radio networksAEU - International Journal of Electronics and Communications, 2013
- Routing in cognitive radio networks: Challenges and solutionsAd Hoc Networks, 2010
- Mobility-based multicast routing algorithm for wireless mobile Ad-hoc networks: A learning automata approachComputer Communications, 2010
- If multi-agent learning is the answer, what is the question?Artificial Intelligence, 2007
- ${Q}$-Learning Algorithms for Constrained Markov Decision Processes With Randomized Monotone Policies: Application to MIMO Transmission ControlIEEE Transactions on Signal Processing, 2007
- Cooperative Multi-Agent Learning: The State of the ArtAutonomous Agents and Multi-Agent Systems, 2005
- Expertness based cooperative Q-learningIEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics), 2002
- A Simple Adaptive Procedure Leading to Correlated EquilibriumEconometrica, 2000
- A dynamic channel assignment policy through Q-learningIEEE Transactions on Neural Networks, 1999
- Individual learning of coordination knowledgeJournal of Experimental & Theoretical Artificial Intelligence, 1998