A Novel Hybrid-ARPPO Algorithm for Dynamic Computation Offloading in Edge Computing
- 6 July 2022
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Internet of Things Journal
- Vol. 9 (23) , 24065-24078
Abstract
Applications consisting of a group of modular tasks can be offloaded to the multiaccess edge computing (MEC) for lower delay and energy consumption. In a dynamic MEC system, the fine-grained cooperative and dynamic offloading solution is necessary for the scenario of reusing tasks among devices. Considering the transmission cooperation, shared wireless bandwidth, and changing task queues on devices and edge servers, we formulate a joint offloading optimization problem to minimize the long-term average task execution cost. Although deep reinforcement learning (DRL) is a popular method for the dynamic problem, existing DRL algorithms are not suitable for our problem because of the hybrid discrete-continuous action spaces and constraints among action dimensions. Therefore, we propose a hybrid average reward proximal policy optimization (hybrid-ARPPO) algorithm to jointly optimize the offloading decisions, cooperative transmission ratios, and edge server assignments. First, we decompose our offloading problem into two subproblems. One is a tractable linear programming problem for continuous transmission ratios, and the other is a Markov decision process (MDP) only with discrete actions for offloading decisions and server assignments. Second, we take the expected average reward as the performance measure and deprecate the discount factor, which can reduce the work of tuning algorithms. Third, we design an action mask layer in the policy network of hybrid-ARPPO to filter invalid actions. Extensive experiments show the effectiveness of our hybrid-ARPPO in different system scales and task arrival patterns.Keywords
Funding Information
- National Key Research and Development Program of China (2018YFB2100300)
- National Natural Science Foundation of China (62172051, 61877005)
This publication has 47 references indexed in Scilit:
- Task Offloading for Mobile Edge Computing in Software Defined Ultra-Dense NetworkIEEE Journal on Selected Areas in Communications, 2018
- ULOOF: A User Level Online Offloading Framework for Mobile Edge ComputingIEEE Transactions on Mobile Computing, 2018
- A Survey on Mobile Edge Computing: The Communication PerspectiveIEEE Communications Surveys & Tutorials, 2017
- Dynamic Computation Offloading for Mobile-Edge Computing With Energy Harvesting DevicesIEEE Journal on Selected Areas in Communications, 2016
- mCloud: A Context-Aware Offloading Framework for Heterogeneous Mobile CloudIEEE Transactions on Services Computing, 2015
- Ad Hoc Cloudlet Based Cooperative Cloud Gaming2014
- A Dynamic Offloading Algorithm for Mobile ComputingIEEE Transactions on Wireless Communications, 2012
- ThinkAir: Dynamic resource allocation and parallel execution in the cloud for mobile code offloading2012
- Joint subcarrier and power allocation in uplink OFDMA systemsIEEE Communications Letters, 2005
- Function Approximation via Tile Coding: Automating Parameter Choice2005