Caching in Dynamic IoT Networks by Deep Reinforcement Learning
- 23 June 2020
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Internet of Things Journal
- Vol. 8 (5) , 3268-3275
Abstract
The sensing service of internet of things (IoT) networks enables IoT sensors to sense the environment information (e.g., temperature and traffic conditions) and send them through IoT gateway to the users who request those information. The explosive growth of IoT users and sensors injects massive traffic into IoT networks and easily depletes the battery of IoT sensors. Caching at the IoT gateway is hence a promising solution to mitigate this problem by storing popular IoT data at the IoT gateway and sending them directly to the users instead of activating IoT sensors to transmit the data. In our work, we investigate the content placement problem, which determines data to be cached at each time epoch in dynamic IoT networks with the objective to minimize the average data transmission delay constrained by the cache storage capacity and IoT data freshness. We formulate our problem as an integer linear programming (ILP) problem and then model it as a Markov decision process (MDP). A deep reinforcement learning algorithm is proposed to solve this problem and its performances are demonstrated via extensive simulations.Keywords
This publication has 34 references indexed in Scilit:
- Optimal and Autonomous Control Using Reinforcement Learning: A SurveyIEEE Transactions on Neural Networks and Learning Systems, 2017
- Dynamic Resource Caching in the IoT Application Layer for Smart CitiesIEEE Internet of Things Journal, 2017
- EdgeIoT: Mobile Edge Computing for the Internet of ThingsIEEE Communications Magazine, 2016
- Resource Management with Deep Reinforcement Learning2016
- Internet of Things: A Survey on Enabling Technologies, Protocols, and ApplicationsIEEE Communications Surveys & Tutorials, 2015
- Energy-Efficiency Oriented Traffic Offloading in Wireless Networks: A Brief Survey and a Learning Approach for Heterogeneous Cellular NetworksIEEE Journal on Selected Areas in Communications, 2015
- Size-Aware Cache Management for Compressed Cache ArchitecturesIEEE Transactions on Computers, 2014
- A Survey of Actor-Critic Reinforcement Learning: Standard and Natural Policy GradientsIEEE Transactions on Systems, Man and Cybernetics, Part C (Applications and Reviews), 2012
- An analysis of temporal-difference learning with function approximationIEEE Transactions on Automatic Control, 1997
- The Complexity of Markov Decision ProcessesMathematics of Operations Research, 1987