ZGZhen Gao

Zhen Gao

*This is an algorithmically generated author record based on publicly available information.

Current Organizations / Institutes

The organizations/institutes from the scholar's latest article

Previous Organizations / Institutes

The organizations / institutes from the scholar's other articles in the past ten years


Zhen Gao has a total of 13 co-authors. Here are the top co-authors by number of shared publications:


Latest Publications

  • CPID-MAAC: RL for Joint User Association and Trajectory Control in UAV-Assisted MEC Systems

    by Zhen Gao, Qipeng Xie, Lei Yang, Yu Dai, Weizheng Wang

    Existing joint user association and trajectory control (JUATC) methods provide remarkably high data rates for mobile users (MUs) in unmanned aerial vehicle (UAV)-assisted multi-access edge computing systems. Nevertheless, current methods give more attention to the uplink and downlink of MU, which must be associated with the same UAV or base station (BS), ignoring the network’s heterogeneity and considerably reducing the MU’s communication efficiency. Furthermore, UAVs typically provide communication services to MUs under partial observation, leading to challenges in achieving optimal service performance due to information loss. Moreover, although existing solutions can readily reach optimal, restriction-fulfilling strategies, they frequently breach restrictions during intermediate iterations. To address these issues, we present a fully decentralized JUATC algorithm based on the Communication and Proportional-Integral-Derivative (PID) Lagrangian-based Multi-Agent Actor-Critic (CPID-MAAC). First, to improve communication efficiency, we consider that each MU can be associated with a different UAV or BS in the uplink and downlink. Second, we establish a messaging mechanism between UAVs based on autoencoding UAV’s observations to handle the information loss. Finally, to alleviate constraint-violating behavior, we incorporate the PID Lagrangian algorithm. The experiments show that CPID-MAAC improves data rate by 7.88%~16.03% and drastically reduces the number of constraint violations during UAV agent training.


  • CSMAAC: Multi-Agent Reinforcement Learning Based Flight Control in Partially Observable Multi-UAV Assisted Crowd Sensing Systems

    by Zhen Gao, Gang Wang, Lei Yang, Chenhao Ying

    In mobile crowd sensing systems, existing flight control methods enable uncrewed aerial vehicles (UAVs) to provide high-quality data collection services for various applications. However, due to limited communication range, UAVs typically collect data under partial observability, hindering optimal performance without global environmental information. Additionally, many methods fail to enforce critical safety constraints. This paper proposes a communication-assisted safe multi-agent actor-critic-based UAV flight control method (CSMAAC). First, we propose an independent prediction communication partner model to address the partial observability problem. Based on the UAV’s local observation, causal inference is used to obtain prior communication information between UAVs through a feed-forward neural network to help UAVs determine potential communication partners. Second, we utilize a critic-network to predict and quantify inter-UAV influence and determine the necessity of communication. By exchanging necessary information inter-UAV, UAVs can perceive global information, thereby solving the UAV’s partial observability problem and reducing communication overhead. Moreover, we propose a similarity enhancement mechanism to improve the learning efficiency of the model by enhancing the connection between UAV observations and the policies of other UAVs. Finally, we introduce a safety layer to Actor-Network to ensure safe UAV flight. The simulation results show that the proposed method outperforms the baselines.


  • MOIPC-MAAC: Communication-Assisted Multiobjective MARL for Trajectory Planning and Task Offloading in Multi-UAV-Assisted MEC

    by Zhen Gao, Jiaming Fu, Zongming Jing, Yu Dai, Lei Yang

    Existing joint trajectory planning and task offloading (JTPTO) methods provide ultra-low latency services for mobile devices (MDs) in unmanned aerial vehicle (UAV)-assisted mobile edge computing (MEC). However, UAVs typically provide services to MDs under partial observation, leading to challenges in achieving optimal service performance due to information loss. Moreover, the JTPTO problem typically involves multi-objective optimization, which is challenging because the objectives may conflict with each other. In this paper, we present a decentralized JTPTO method based on Multi-Objective and Independently Predicted Communication Multi-Agent Actor-Critic (MOIPCMAAC). First, an IPC network is designed to facilitate UAV agents in learning a prior for communication between UAVs. UAV agents learn this prior through causal reasoning, which represents the mapping of UAV’s observation to the level of confidence in choosing communication partners. The effect of one UAV on another UAV is predicted through the critic-network in multi-agent reinforcement learning (MARL) and measured to indicate the necessity of UAV-UAV communication. Further, we regularize JTPTO policies to more effectively utilize exchanged messages. Second, a generalized variant of the Bellman optimality operator with multiple objectives is applied to address the JTPTO problem. We use it to learn a single parameterized expression that encompasses all the best JTPTO policies across the space of preferences. Experiments show that compared to existing solutions, MOIPC-MAAC reduces system costs by 14.23%~19.56% and the communication cost to approximately 11.23%. Moreover, compared to training from scratch, MOIPC-MAAC accelerates the adaptation of new JTPTO tasks with unknown preferences by 13.12%.


  • MO-AVC: Deep-Reinforcement-Learning-Based Trajectory Control and Task Offloading in Multi-UAV-Enabled MEC Systems

    by Zhen Gao, Lei Yang, Yu Dai

    We investigate the joint trajectory control and task offloading (JTCTO) problem in multi-unmanned aerial vehicle (UAV)-enabled mobile edge computing (MEC). However, existing JTCTO solutions primarily focus on fixed UAV-enabled MEC scenario variations and necessitate extensive interaction to adapt to new scenarios. Moreover, we consider minimizing task latency and UAV’s energy consumption, and maximizing the quantity of tasks collected by the UAV as optimization goals. However, this optimization problem is characterized by multiple conflicting goals that should be adjusted according to their relative significance. In this paper, we present a Multi-Objective Actor-Variations Critic-based JTCTO solution (MO-AVC). First, a group of reinforcement learning strategies is utilized to collect experience on training scenarios, which are employed to learn embeddings of both strategies and scenarios. Further, these two embeddings are used as inputs to train the AVC, which explicitly estimates the total return in a space of JTCTO strategies and UAV-enabled MEC scenarios. When adapting to a new scenario, just a few steps of scenario interaction are enough to predict the scenario embedding, thus selecting strategies by maximizing the trained AVC. Second, we propose an actor-conditioned critic framework where the outputs are conditioned on the varying significance of goals, and present a weight dynamic memory-based experience replay to address the intrinsic instability of the dynamic weight context. Finally, simulation results show that MO-AVC can quickly adapt to new scenarios. Moreover, MO-AVC reduces the latency by 7.56%~10.57%, the energy consumption by 11.11%~17.27%, and increases the tasks number by 10.33%~15.54% compared to existing solutions.


  • Large-Scale Cooperative Task Offloading and Resource Allocation in Heterogeneous MEC Systems via Multiagent Reinforcement Learning

    by Zhen Gao, Lei Yang, Yu Dai

    In multi-access edge computing systems, existing task offloading methods have provided ultra-short latency services for heterogeneous tasks on mobile devices (MDs). Nevertheless, the complexity of MEC systems grows exponentially with the number of MDs or edge servers (ES), so learning a good offloading policy is a huge challenge when the number of MDs or ESs is large. Moreover, MDs are often unable to find optimal ESs for offloading since the restricted ESs infrastructures and the spatiotemporally imbalanced task offloading requirements. To solve these problems, we propose a Curriculum Spatio-Temporal Multi-Agent Actor-Critic (CSTMAAC)-based task offloading method. Each ES is regarded as an agent and the problem is formulated as a multi-objective optimization task. To adapt to the large-scale MEC systems, we first introduce an evolutionary curriculum learning by gradually raising the number of trained ES agents in a phased way. Second, to facilitate the coordination of the offloading policies among geographically distributed ESs, we design an attention-based centralized critic-network. Besides, a delayed access mechanism is introduced that uses information about future task processing competition to capture the impact of potential future task processing contention and help ES agents obtain a better offloading strategy. Finally, critic-network is expanded to multi-critics and a dynamic weight mechanism is designed to adaptively optimize multi-objectives and obtain a good balance for multiple objectives. Real-world datasets used in experiments demonstrate that CSTMAAC raises task completion rates and total utility by 13.01% 15.21% and 16.89% 18.32% compared with the existing algorithms.


  • Fast Adaptive Task Offloading and Resource Allocation in Large-Scale MEC Systems via Multiagent Graph Reinforcement Learning

    by Zhen Gao, Lei Yang, Yu Dai

    In multi-access edge computing, when many mobile devices (MDs) offload their tasks to an edge server (ES), its resources might become constrained. These tasks may take a long time to complete or even be thrown away. Since the unknown information of both the ESs and other MDs, it is difficult for each MD to determine its offloading policy independently. Furthermore, most offloading methods have poor generalization to new environment since they focus on model architecture with a fixed quantity of MDs and ESs, preventing trained models from transferring to other environments. In the paper, we provide a full decentralized offloading scheme based on the Curriculum Attention-weighted Graph Recurrent Network-based Multi-Agent Actor-Critic (CAGR-MAAC). First, we build MEC as a shared MD agents-ESs graph and an AGR-based message network is designed to enable each MD aggregate the information of ESs and other MDs and solve the partial observability of MD agents for MEC system. Second, a learnable differentiable encoder network is introduced to construct MD agent’s local information encoding. Subsequently, the MD agent converts overall the information regarding the MEC system into a fixed-size embedding via an AGR Network to handle different quantity of MDs and ESs. Finally, we introduce curriculum learning to address the huge complexity of the MEC system and the training difficulties induced by the large amounts of MDs and ESs. Experiments demonstrate that compared with existing algorithms, CAGR-MAAC boosts task completion rates and decreases system costs by 13.01% 15.03% and 16.45% 18.56%, and can quickly adapt to the new environment.


  • Fast Adaptive Task Offloading and Resource Allocation via Multiagent Reinforcement Learning in Heterogeneous Vehicular Fog Computing

    by Zhen Gao, Lei Yang, Yu Dai

    In vehicular fog computing, task offloading enables mobile vehicles (MVs) to offer ultralow latency services for computation-intensive tasks. Nevertheless, the edge server (ES) may have a high load when a large number of MVs offload their tasks to it, causing many tasks either experience long processing times or being dropped, particularly for latency-sensitive tasks. Moreover, most existing methods are largely limited to training a model from scratch for new environments. This is because they focus more on model structures with fixed input and output sizes, impeding the transfer of trained models across different environments. To solve these problems, we propose a decentralized task offloading method based on transformer and policy decoupling-based multiagent actor–critic (TPDMAAC). We first introduce a transformer-based long sequence forecasting network (TLSFN) for predicting the current and future queuing delay of ESs to solve uncertain load. Second, we redesign the actor-network using transformer-based temporal feature extraction network (TTFEN) and policy decoupling network (PDN). TTFEN can adapt to various input sizes through a transformer that accepts different tokens we build from the raw input. PDN provides a mapping between the transformer-based embedding features and offloading policies utilizing self-attention mechanism to address various output dimensions. Finally, the experiments on two real-world data sets show that TPDMAAC can quickly adapt to a new environment. And compared to existing algorithms, TPDMAAC reduces the system cost by 11.01%–12.03% as well as improves task completion rates by 10.45%–13.56%.


  • Top Cited Papers

    Large-Scale Computation Offloading Using a Multi-Agent Reinforcement Learning in Heterogeneous Multi-Access Edge Computing

    by Zhen Gao, Lei Yang, Yu Dai

    Recently, existing computation offloading methods have provided extremely low service latency for mobile users (MUs) in multi-access edge computing (MEC). However, this remains a challenge in large-scale mixed cooperative-competitive MUs heterogeneous MEC environments. Moreover, existing methods focus more on all offloaded tasks handled by static resource allocation MEC servers (ESs) within a time interval, ignoring on-demand requirements of heterogeneous tasks, resulting in many tasks being dropped or wasting resources, especially for latency-sensitive tasks. To address these issues, we present a decentralized computation offloading solution based on the Attention-weighted Recurrent Multi-Agent Actor-Critic (ARMAAC). First, we design a recurrent actor-critic framework to assist MU agents in remembering historical resource allocation information of ESs to better understand the future state of ESs, especially in dynamic resource allocation. Second, an attention mechanism is introduced to compress the joint observation space dimension of all MUs agent to adapt to large-scale MUs. Finally, the actor-critic framework with double centralized critics and Dueling network is redesigned considering the instability and convergence difficulties caused by the sensitive relationship between the actor and critic networks. The experiments show that ARMAAC improves task completion rates and reduces average system cost by 11.01% $\sim$ 14.03% and 10.45% $\sim$ 15.56% compared with baselines.