Self-Organization in Small Cell Networks: A Reinforcement Learning Approach
- 18 June 2013
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Wireless Communications
- Vol. 12 (7) , 3202-3212
Abstract
In this paper, a decentralized and self-organizing mechanism for small cell networks (such as micro-, femto- and picocells) is proposed. In particular, an application to the case in which small cell networks aim to mitigate the interference caused to the macrocell network, while maximizing their own spectral efficiencies, is presented. The proposed mechanism is based on new notions of reinforcement learning (RL) through which small cells jointly estimate their time-average performance and optimize their probability distributions with which they judiciously choose their transmit configurations. Here, a minimum signal to interference plus noise ratio (SINR) is guaranteed at the macrocell user equipment (UE), while the small cells maximize their individual performances. The proposed RL procedure is fully distributed as every small cell base station requires only an observation of its instantaneous performance which can be obtained from its UE. Furthermore, it is shown that the proposed mechanism always converges to an epsilon Nash equilibrium when all small cells share the same interest. In addition, this mechanism is shown to possess better convergence properties and incur less overhead than existing techniques such as best response dynamics, fictitious play or classical RL. Finally, numerical results are given to validate the theoretical findings, highlighting the inherent tradeoffs facing small cells, namely exploration/exploitation, myopic/foresighted behavior and complete/incomplete information.Keywords
This publication has 28 references indexed in Scilit:
- Distributed Learning Strategies for Interference Mitigation in Femtocell Networks2011
- Enhanced intercell interference coordination challenges in heterogeneous networksIEEE Wireless Communications, 2011
- Decentralized Cross-Tier Interference Mitigation in Cognitive Femtocell Networks2011
- On the Global Convergence of Stochastic Fictitious PlayEconometrica, 2002
- Potential GamesGames and Economic Behavior, 1996
- Learning Dynamics in Games with Stochastic PerturbationsGames and Economic Behavior, 1995
- Quantal Response Equilibria for Normal Form GamesGames and Economic Behavior, 1995
- Decentralized learning of Nash equilibria in multi-person stochastic games with incomplete informationIEEE Transactions on Systems, Man, and Cybernetics, 1994
- Information Theory and Statistical MechanicsPhysical Review B, 1957
- Equilibrium points in n -person gamesProceedings of the National Academy of Sciences, 1950