Hierarchical Multi-Agent Deep Reinforcement Learning for Backscatter-aided Data Offloading

Authors:
Publisher website
Google Scholar
Abstract
In this paper, we consider a hybrid computation offloading scheme that allows edge users to offload workloads to the edge servers by using active RF communications and backscatter communications. We aim to maximize the overall energy efficiency by jointly optimizing the beamforming of access point (AP) and the users’ offloading decisions. Considering a dynamic environment, we propose a hierarchical multi-agent deep reinforcement learning (H-MADRL) framework to solve this problem. The high-level agent resides in the AP and optimizes the beamforming strategy, while the low-level user agents learn and adapt individuals’ offloading strategies. To further improve the learning efficiency, we propose a novel optimization-driven learning algorithm that allows the AP to estimate the low-level users’ actions by solving an approximate problem efficiently. Then, the action estimation can be shared with all users and drive them to update individuals’ actions independently. Simulation results reveal that our algorithm can improve the reward performance by 50%. The learning efficiency and reliability are also enhanced comparing to the conventional model-free learning methods.
Funding Information
  • National Natural Science Foundation of China
  • Shenzhen Municipal Fundamental Research Program
  • National Science Foundation

This publication has 14 references indexed in Scilit: