Joint Computation Offloading and Service Caching in Mobile Edge-Cloud Computing via Deep Reinforcement Learning
- 30 August 2024
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Internet of Things Journal
- Vol. 11 (24) , 40331-40344
Abstract
Mobile edge computing (MEC) emerges as a promising paradigm that aims to extend the computing capabilities of mobile devices. This enhancement is achieved by offloading heavy computational tasks to nearby edge servers, where relevant services, such as programs, databases, and libraries, are stored to support the execution of these tasks. Given the restricted storage capacity and computing resources of edge servers, it is crucial to carefully consider which tasks to offload and which services to cache. However, the highly dynamic system and closely coupled decisions between offloading and caching pose considerable challenges to developing an effective strategy. In this article, we investigate both computation offloading and service caching in an end-edge–cloud collaborative system under time-varying wireless channels and task requests. We formulate an optimization problem aimed at minimizing the system cost, including time and energy consumption, and model it further as a Markov decision process (MDP). To address this, we propose a deep reinforcement learning (DRL)-based approach with dynamic action masking to concurrently optimize offloading and caching decisions. This approach renormalizes the valid action probability distribution at every step, enabling the agent to explore within the space of valid action combinations and thereby learn the optimal policy. Extensive experimental results demonstrate that the proposed approach greatly reduces the system cost compared to other representative benchmark schemes.Keywords
Funding Information
- National Natural Science Foundation of China (62172051, 62272052)
This publication has 40 references indexed in Scilit:
- Deep-Reinforcement-Learning-Based Joint Caching and Resources Allocation for Cooperative MECIEEE Internet of Things Journal, 2023
- Energy-Efficient Resource Allocation for Heterogeneous Edge–Cloud ComputingIEEE Internet of Things Journal, 2023
- Edge Computing for Internet of Everything: A SurveyIEEE Internet of Things Journal, 2022
- Request Scheduling Combined With Load Balancing in Mobile-Edge ComputingIEEE Internet of Things Journal, 2022
- Resource Scheduling in Edge Computing: A SurveyIEEE Communications Surveys & Tutorials, 2021
- Gym-µRTS: Toward Affordable Full Game Real-time Strategy Games Research with Deep Reinforcement Learning2021
- Deep Reinforcement Learning for Energy-Efficient Computation Offloading in Mobile-Edge ComputingIEEE Internet of Things Journal, 2021
- Partial Computation Offloading in NOMA-Assisted Mobile-Edge Computing Systems Using Deep Reinforcement LearningIEEE Internet of Things Journal, 2021
- Convergence of Edge Computing and Deep Learning: A Comprehensive SurveyIEEE Communications Surveys & Tutorials, 2020
- A Mathematical Theory of CommunicationBell System Technical Journal, 1948