Distributed DRL-Based Computation Offloading Scheme for Improving QoE in Edge Computing Environments
Open Access
Abstract
Various edge collaboration schemes that rely on reinforcement learning (RL) have been proposed to improve the quality of experience (QoE). Deep RL (DRL) maximizes cumulative rewards through large-scale exploration and exploitation. However, the existing DRL schemes do not consider the temporal states using a fully connected layer. Moreover, they learn the offloading policy regardless of the importance of experience. They also do not learn enough because of their limited experiences in distributed environments. To solve these problems, we proposed a distributed DRL-based computation offloading scheme for improving the QoE in edge computing environments. The proposed scheme selects the offloading target by modeling the task service time and load balance. We implemented three methods to improve the learning performance. Firstly, the DRL scheme used the least absolute shrinkage and selection operator (LASSO) regression and attention layer to consider the temporal states. Secondly, we learned the optimal policy based on the importance of experience using the TD error and loss of the critic network. Finally, we adaptively shared the experience between agents, based on the strategy gradient, to solve the data sparsity problem. The simulation results showed that the proposed scheme achieved lower variation and higher rewards than the existing schemes.Keywords
Funding Information
- Korean government (MSIT) (2020-0-00998)
- Research Grant of Kwangwoon University (2020-0-00998)
This publication has 29 references indexed in Scilit:
- Edge computing: A surveyFuture Generation Computer Systems, 2019
- A Heuristic Offloading Method for Deep Learning Edge Services in 5G NetworksIEEE Access, 2019
- Edge Cloud Offloading AlgorithmsACM Computing Surveys, 2019
- Task Proactive Caching Based Computation Offloading and Resource Allocation in Mobile-Edge Computing Systems2018
- QoE-Based Big Data Analysis with Deep Learning in Pervasive Edge Environment2018
- Fair Task Offloading among Fog Nodes in Fog Computing Networks2018
- Locality-Aware Load Sharing in Mobile Cloud Computing2017
- Efficient Multi-User Computation Offloading for Mobile-Edge Cloud ComputingIEEE/ACM Transactions on Networking, 2015
- Regression Shrinkage and Selection via The Lasso: A RetrospectiveJournal of the Royal Statistical Society Series B: Statistical Methodology, 2011
- Pervasive computing: vision and challengesIEEE Wireless Communications, 2001