Multi-Agent Reinforcement Learning for Efficient Content Caching in Mobile D2D Networks
Authors:
Top Cited Papers
- 29 January 2019
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Wireless Communications
- Vol. 18 (3) , 1610-1622
Abstract
To address the increase of multimedia traffic dominated by streaming videos, user equipment (UE) can collaboratively cache and share contents to alleviate the burden of base-stations. Prior work on D2D caching policies assumes perfect knowledge of the content popularity distribution. Since the content popularity distribution is usually unavailable in advance, a machine learning based caching strategy that exploits the knowledge of content demand history would be highly promising. Thus, we design D2D caching strategies using multi-agent reinforcement learning in this paper. Specifically, we model the D2D caching problem as a multi-agent multi-armed bandit problem, and use Q-learning to learn how to coordinate the caching decisions. UEs can be independent learners (ILs) if they learn the Qvalues of their own actions, and joint action learners (JALs) if they learn the Q-values of their own actions in conjunction with those of other UEs. As the action space is very large leading to high computational complexity, a modified combinatorial upper confidence bound algorithm is proposed to reduce the action space for both IL and JAL. Simulation results show that the proposed JAL-based caching scheme outperforms IL-based caching scheme and other popular caching schemes in terms of average downloading latency and cache hit rate.Keywords
Funding Information
- National Natural Science Foundation of China (61631005, 61871099)
- National Science Foundation (CNS-1526425, CNS-1815465)
This publication has 33 references indexed in Scilit:
- A transfer learning approach for cache-enabled wireless networks2015
- Mobile device-to-device (D2D) content delivery networking: A design and optimization frameworkJournal of Communications and Networks, 2014
- Living on the edge: The role of proactive caching in 5G wireless networksIEEE Communications Magazine, 2014
- Device-to-device communications in cellular networksIEEE Communications Magazine, 2014
- A survey on content-centric technologies for the current Internet: CDN and P2P solutionsComputer Communications, 2012
- Exact and greedy solutions of the knapsack problem: the ratio of values of objective functionsJournal of Computer and Systems Sciences International, 2010
- Reinforcement Learning Approaches to Coordination in Cooperative Multi-agent Systems2003
- Evaluating content management techniques for Web proxy cachesACM SIGMETRICS Performance Evaluation Review, 2000
- Removal policies in network caches for World-Wide Web documentsACM SIGCOMM Computer Communication Review, 1996
- Non-Cooperative GamesAnnals of Mathematics, 1951