Han Zhang
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Research Subjects
Han Zhang has a total of 18 co-authors. Here are the top co-authors by number of shared publications:
- Hongbin Liang
Southwest Jiaotong University
6 shared publications - Dongmei Zhao
Department of Electrical and Computer Engineering, McMaster University, Hamilton, ON, Canada
5 shared publications - Guotao Mao
School of Transportation and Logistics, the National United Engineering Laboratory of Integrated and Intelligent Transportation, and the National Engineering Laboratory of Integrated Transportation Big Data Application Technology, Southwest Jiaotong University, Chengdu, China
3 shared publications - Lei Wang
School of Transportation and Logistics, the National United Engineering Laboratory of Integrated and Intelligent Transportation, and the National Engineering Laboratory of Integrated Transportation Big Data Application Technology, Southwest Jiaotong University, Chengdu, China
3 shared publications - Xintao Hong
School of Information Science and Technology, Southwest Jiaotong University, Chengdu, China
3 shared publications Yiting Yao
School of Transportation and Logistics, National United Engineering Laboratory of Integrated and Intelligent Transportation, and National Engineering Laboratory of Integrated Transportation Big Data Application Technology, Southwest Jiaotong University, Chengdu, China
3 shared publications
Latest Publications
- 25 August 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Vehicular Technology
- Vol. 75 (2) , 2209-2223
Attention-Based Deep Reinforcement Learning for Joint Trajectory Planning and Task Offloading in AAV-Assisted Vehicular Edge Computing
The rapid growth of compute-intensive and delay-sensitive applications in the Internet of Vehicles (IoV) has brought new challenges to traditional Vehicle Edge Computing (VEC). Especially during peak traffic periods, fixed base stations (BSs), due to their static deployment and limited coverage, struggle to elastically scale computing resources to meet the dynamically changing computational demands of vehicular users, which to some extent affects the overall quality of service and efficiency of system resource utilization. In this paper, we propose a hybrid edge computing framework that integrates Autonomous Aerial Vehicles (AAVs) as mobile computing nodes to complement fixed base stations. For this AAV-BS hybrid edge computing environment, we design and optimize a computation offloading model considering heterogeneous task types and AAV trajectory planning. The proposed model tackles the joint optimization of system revenue, service delay, and energy consumption, while considering practical constraints such as AAV energy limitations and base station capacity. To address this multi-objective optimization challenge, we construct a Markov Decision Process (MDP) model and develop a multi-head self-attention enhanced Multi-Agent Deep Dirichlet Deterministic Policy Gradient (MHSA-MAD3PG) algorithm. The multi-head self-attention mechanism enables agents to capture complex dependencies and interactions in the environment, leading to more effective collaborative decision-making. Comprehensive simulation results demonstrate that our proposed approach achieves superior performance in terms of system revenue, service delay, and energy efficiency compared to baseline methods.
- 25 August 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Vehicular Technology
- Vol. 75 (2) , 2209-2223
Attention-Based Deep Reinforcement Learning for Joint Trajectory Planning and Task Offloading in AAV-Assisted Vehicular Edge Computing
The rapid growth of compute-intensive and delay-sensitive applications in the Internet of Vehicles (IoV) has brought new challenges to traditional Vehicle Edge Computing (VEC). Especially during peak traffic periods, fixed base stations (BSs), due to their static deployment and limited coverage, struggle to elastically scale computing resources to meet the dynamically changing computational demands of vehicular users, which to some extent affects the overall quality of service and efficiency of system resource utilization. In this paper, we propose a hybrid edge computing framework that integrates Autonomous Aerial Vehicles (AAVs) as mobile computing nodes to complement fixed base stations. For this AAV-BS hybrid edge computing environment, we design and optimize a computation offloading model considering heterogeneous task types and AAV trajectory planning. The proposed model tackles the joint optimization of system revenue, service delay, and energy consumption, while considering practical constraints such as AAV energy limitations and base station capacity. To address this multi-objective optimization challenge, we construct a Markov Decision Process (MDP) model and develop a multi-head self-attention enhanced Multi-Agent Deep Dirichlet Deterministic Policy Gradient (MHSA-MAD3PG) algorithm. The multi-head self-attention mechanism enables agents to capture complex dependencies and interactions in the environment, leading to more effective collaborative decision-making. Comprehensive simulation results demonstrate that our proposed approach achieves superior performance in terms of system revenue, service delay, and energy efficiency compared to baseline methods.
- 9 January 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Internet of Things Journal
- Vol. 12 (11) , 15453-15466
Joint Task Partitioning and Resource Allocation in RAV-Enabled Vehicular Edge Computing Based on Deep Reinforcement Learning
Vehicle Edge Computing (VEC) leverages compact cloud computing at the mobile network edge to meet the processing and latency needs of vehicles. By bringing computation closer to the vehicles, VEC reduces data transmission, minimizes latency, and boosts performance for compute-intensive applications. However, during peak hours of urban road traffic, the scarce computational resources available at edge servers could pose challenges in fulfilling the processing needs of vehicles. Introducing Unmanned Aerial Vehicles (UAVs) as supplementary edge computing nodes could significantly mitigate the aforementioned issue. In this paper, we propose a flexible edge computing framework in which a fleet of UAVs function as mobile computational service providers, offering computation offloading services to multiple vehicles. We design and optimize a computation offloading model for the UAV-enabled vehicle edge computing environment. The proposed model tackles the task offloading challenge, aiming to optimize UAV revenue and task processing efficiency while considering the constraints of UAVs’ restricted computational power and energy resources. Towards this end, our model jointly considers two key factors: task partitioning and computational resource allocation. To tackle the challenges posed by the aforementioned non-convex optimization problem, we construct a Markov Decision Process (MDP) model for the multi-UAV-enabled mobile edge computing system and introduce an innovative Multi-Agent Deep Reinforcement Learning (MADRL) framework addressing the decision-making challenge represented by MDP model. Comprehensive simulation outcomes illustrate that our devised task offloading technique outperforms other optimization methods.
- 9 January 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Internet of Things Journal
- Vol. 12 (11) , 15453-15466
Joint Task Partitioning and Resource Allocation in RAV-Enabled Vehicular Edge Computing Based on Deep Reinforcement Learning
Vehicle Edge Computing (VEC) leverages compact cloud computing at the mobile network edge to meet the processing and latency needs of vehicles. By bringing computation closer to the vehicles, VEC reduces data transmission, minimizes latency, and boosts performance for compute-intensive applications. However, during peak hours of urban road traffic, the scarce computational resources available at edge servers could pose challenges in fulfilling the processing needs of vehicles. Introducing Unmanned Aerial Vehicles (UAVs) as supplementary edge computing nodes could significantly mitigate the aforementioned issue. In this paper, we propose a flexible edge computing framework in which a fleet of UAVs function as mobile computational service providers, offering computation offloading services to multiple vehicles. We design and optimize a computation offloading model for the UAV-enabled vehicle edge computing environment. The proposed model tackles the task offloading challenge, aiming to optimize UAV revenue and task processing efficiency while considering the constraints of UAVs’ restricted computational power and energy resources. Towards this end, our model jointly considers two key factors: task partitioning and computational resource allocation. To tackle the challenges posed by the aforementioned non-convex optimization problem, we construct a Markov Decision Process (MDP) model for the multi-UAV-enabled mobile edge computing system and introduce an innovative Multi-Agent Deep Reinforcement Learning (MADRL) framework addressing the decision-making challenge represented by MDP model. Comprehensive simulation outcomes illustrate that our devised task offloading technique outperforms other optimization methods.
- 26 August 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Network
- Vol. 39 (3) , 249-256
Enhanced DRL Strategy for Distributed Edge Computing in Vehicular Networks
The rapid growth of connected vehicles has posed significant challenges in managing computational offloading in vehicular networks. However, recent advancements in artificial intelligence, big data, and cloud computing have opened up new possibilities for optimizing computational offloading in vehicular edge computing (VEC) systems. This article addresses the critical and pressing issue of leveraging these new technologies to optimize the allocation of vehicular computational offloading resources at edge servers, with the aim of improving the efficiency of computational offloading services while reducing costs. We focus on the computational offloading problem in a distributed edge computing system, where computational tasks from vehicles can be processed by multiple edge computing resource units (ECRUs). To tackle this problem, we first model the optimization of computational offloading resources as a Markov Decision Process (MDP), taking into account system income, cost expenditure, and other relevant domains. We then propose an Enhanced Deep Q-Learning (EDQL) approach, which leverages reinforcement learning (RL) to solve the optimization problem effectively. Our simulation results demonstrate that the proposed EDQL approach achieves fast convergence speed, high convergence stability, and superior performance compared to existing methods.
- 26 August 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Network
- Vol. 39 (3) , 249-256
Enhanced DRL Strategy for Distributed Edge Computing in Vehicular Networks
The rapid growth of connected vehicles has posed significant challenges in managing computational offloading in vehicular networks. However, recent advancements in artificial intelligence, big data, and cloud computing have opened up new possibilities for optimizing computational offloading in vehicular edge computing (VEC) systems. This article addresses the critical and pressing issue of leveraging these new technologies to optimize the allocation of vehicular computational offloading resources at edge servers, with the aim of improving the efficiency of computational offloading services while reducing costs. We focus on the computational offloading problem in a distributed edge computing system, where computational tasks from vehicles can be processed by multiple edge computing resource units (ECRUs). To tackle this problem, we first model the optimization of computational offloading resources as a Markov Decision Process (MDP), taking into account system income, cost expenditure, and other relevant domains. We then propose an Enhanced Deep Q-Learning (EDQL) approach, which leverages reinforcement learning (RL) to solve the optimization problem effectively. Our simulation results demonstrate that the proposed EDQL approach achieves fast convergence speed, high convergence stability, and superior performance compared to existing methods.
- 6 August 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Internet of Things Journal
- Vol. 11 (22) , 37114-37126
DRL-Based Joint Resource Allocation and Platoon Control Optimization for UAV-Hosted Platoon Digital Twin
Digital twin (DT)-empowered platoon can improve platoon management efficiency and driving safety. However, the resource allocation scheme of low-latency platoon digital twin (PDT) and the interactions with platoon control strategy are important issues in the study of PDTs. In this paper, we study the resource allocation in the PDT network and the interaction mechanism between PDT and platoon control for an unmanned aerial vehicle (UAV)-hosted PDT. We introduce the age of information (AoI) metrics to characterize the freshness of the DTs. To explore the impact of the PDT resource allocation scheme on the platoon control strategy, we propose a joint optimization model for power resource allocation and platoon control. Specifically, the allocation of power resources affects the PDT’s AoI, and the high-latency PDT in turn affects the platoon control strategy. Our objective is minimize the weighted sum of the system’s average energy consumption and the PDT’s average peak AoI. To solve the problem, we first reformulate the power resource allocation problem over a period of time as a Markov decision process (MDP) model, and then propose the Dirichlet deep deterministic policy gradient-based power allocation (D3PGPA) method based on Dirichlet distribution and DDPG algorithm. The method can not only effectively explores the state space while satisfying the constraints of limited resources but also improve the stability of the algorithm. Numerical results show that the D3PGPA method can host a PDT with low AoI and improve the stability of the platoon. Besides, our proposed method performs stably and outperforms other benchmark methods.
- 6 August 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Internet of Things Journal
- Vol. 11 (22) , 37114-37126
DRL-Based Joint Resource Allocation and Platoon Control Optimization for UAV-Hosted Platoon Digital Twin
Digital twin (DT)-empowered platoon can improve platoon management efficiency and driving safety. However, the resource allocation scheme of low-latency platoon digital twin (PDT) and the interactions with platoon control strategy are important issues in the study of PDTs. In this paper, we study the resource allocation in the PDT network and the interaction mechanism between PDT and platoon control for an unmanned aerial vehicle (UAV)-hosted PDT. We introduce the age of information (AoI) metrics to characterize the freshness of the DTs. To explore the impact of the PDT resource allocation scheme on the platoon control strategy, we propose a joint optimization model for power resource allocation and platoon control. Specifically, the allocation of power resources affects the PDT’s AoI, and the high-latency PDT in turn affects the platoon control strategy. Our objective is minimize the weighted sum of the system’s average energy consumption and the PDT’s average peak AoI. To solve the problem, we first reformulate the power resource allocation problem over a period of time as a Markov decision process (MDP) model, and then propose the Dirichlet deep deterministic policy gradient-based power allocation (D3PGPA) method based on Dirichlet distribution and DDPG algorithm. The method can not only effectively explores the state space while satisfying the constraints of limited resources but also improve the stability of the algorithm. Numerical results show that the D3PGPA method can host a PDT with low AoI and improve the stability of the platoon. Besides, our proposed method performs stably and outperforms other benchmark methods.
- 14 June 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Vehicular Technology
- Vol. 73 (10) , 15609-15620
Resource Allocation for Dynamic Platoon Digital Twin Networks: A Multi-Agent Deep Reinforcement Learning Method
Vehicle driving in a platoon is an efficient and ecological driving solution. Introducing the concept of digital twin (DT) into the platoon to establish platoon digital twin (PDT) can improve the management efficiency and driving safety of the platoon. However, the joint allocation of multiple types of resources in a platoon digital twin network (PDTN) is an important issue for the successful implementation and maintenance of the PDT. In this paper, we investigate the resource allocation problem in a PDTN. By comprehensively considering the effects of high mobility of platooning vehicles, real-time nature of the DTs, and multi-vehicle cooperation, we propose a PDT utility optimization model for bandwidth and computation resource allocation. We formulate the dynamic resource allocation problem as an M-th order Markov decision process (MDP) and design a deep reinforcement learning (DRL)-based dynamic resource allocation (DRLDRA) method to solve it. To optimize the actions of the agent, we reshape the state in a smaller time granularity to better reflect the temporal variations of the state. Correspondingly, we design temporal feature extraction neural networks (TFENNs) based on multi-head self-attention (MHSA) mechanism and long short-term memory (LSTM) to extract the temporal features of the state. To improve the learning efficiency, a decentralized multi-agent deep deterministic policy gradient (DDPG)-based learning framework is proposed. Numerical results show that the DRLDRA method performs excellently and outperforms other benchmark methods.
- 14 June 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Vehicular Technology
- Vol. 73 (10) , 15609-15620
Resource Allocation for Dynamic Platoon Digital Twin Networks: A Multi-Agent Deep Reinforcement Learning Method
Vehicle driving in a platoon is an efficient and ecological driving solution. Introducing the concept of digital twin (DT) into the platoon to establish platoon digital twin (PDT) can improve the management efficiency and driving safety of the platoon. However, the joint allocation of multiple types of resources in a platoon digital twin network (PDTN) is an important issue for the successful implementation and maintenance of the PDT. In this paper, we investigate the resource allocation problem in a PDTN. By comprehensively considering the effects of high mobility of platooning vehicles, real-time nature of the DTs, and multi-vehicle cooperation, we propose a PDT utility optimization model for bandwidth and computation resource allocation. We formulate the dynamic resource allocation problem as an M-th order Markov decision process (MDP) and design a deep reinforcement learning (DRL)-based dynamic resource allocation (DRLDRA) method to solve it. To optimize the actions of the agent, we reshape the state in a smaller time granularity to better reflect the temporal variations of the state. Correspondingly, we design temporal feature extraction neural networks (TFENNs) based on multi-head self-attention (MHSA) mechanism and long short-term memory (LSTM) to extract the temporal features of the state. To improve the learning efficiency, a decentralized multi-agent deep deterministic policy gradient (DDPG)-based learning framework is proposed. Numerical results show that the DRLDRA method performs excellently and outperforms other benchmark methods.
- 20 February 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Vehicular Technology
- Vol. 73 (7) , 9591-9603
DRL-Based Resource Allocation Game With Influence of Review Information for Vehicular Edge Computing Systems
Vehicle Edge Computing (VEC) represents a new technological paradigm. It delivers computational resources via edge nodes situated close to users. This approach not only satisfies the growing computational needs of vehicles but also minimizes communication latency. Such advancements are crucial for the evolution of intelligent transportation systems. To ensure these systems succeed, two key strategies are essential. First, edge servers must be effectively incentivized to engage in computation offloading. Second, vehicles require efficient resource request strategies, particularly when edge resources are limited. In this paper, we consider a duopolistic edge service market for vehicles with the existence of two service stage. For edge servers, they announce their resource pricing strategies before the start of each stage after a game has been played. After the conclusion of the first stage, vehicles generate reviews based on their service experience for both servers. These reviews will affect the vehicle's choice of edge servers in the next stage. Therefore, edge servers must devise effective pricing strategies to optimize their profits over both stages. Vehicles, after making their choice at any stage based on personal preferences, service quality, and resource pricing, must also engage in a game with other vehicles choosing the same server to determine their resource request strategy. In cases where vehicles prefer not to disclose their resource requests and other information, we propose a deep reinforcement learning framework to maximize the utility of each vehicle. Simulation results validate the effectiveness of our resource allocation scheme based on game theory and deep reinforcement learning.
- 20 February 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Vehicular Technology
- Vol. 73 (7) , 9591-9603
DRL-Based Resource Allocation Game With Influence of Review Information for Vehicular Edge Computing Systems
Vehicle Edge Computing (VEC) represents a new technological paradigm. It delivers computational resources via edge nodes situated close to users. This approach not only satisfies the growing computational needs of vehicles but also minimizes communication latency. Such advancements are crucial for the evolution of intelligent transportation systems. To ensure these systems succeed, two key strategies are essential. First, edge servers must be effectively incentivized to engage in computation offloading. Second, vehicles require efficient resource request strategies, particularly when edge resources are limited. In this paper, we consider a duopolistic edge service market for vehicles with the existence of two service stage. For edge servers, they announce their resource pricing strategies before the start of each stage after a game has been played. After the conclusion of the first stage, vehicles generate reviews based on their service experience for both servers. These reviews will affect the vehicle's choice of edge servers in the next stage. Therefore, edge servers must devise effective pricing strategies to optimize their profits over both stages. Vehicles, after making their choice at any stage based on personal preferences, service quality, and resource pricing, must also engage in a game with other vehicles choosing the same server to determine their resource request strategy. In cases where vehicles prefer not to disclose their resource requests and other information, we propose a deep reinforcement learning framework to maximize the utility of each vehicle. Simulation results validate the effectiveness of our resource allocation scheme based on game theory and deep reinforcement learning.
- Open Access
- 8 February 2024
- Published by Institution of Engineering and Technology (IET) in IET Intelligent Transport Systems
- Vol. 18 (6) , 1109-1120
Joint resource allocation and security redundancy for autonomous driving based on deep reinforcement learning algorithm
Autonomous vehicles navigating urban roads require technology that combines low latency with high computing power. The limited resources of the vehicle itself compel it to offload task requirements to edge server (ES) for processing assistance. However, as the number of vehicles continues to increase, how edge servers reasonably allocate limited resources to autonomous vehicles becomes critical to the success of urban intelligent transportation services. This paper establishes an urban road scenario with multiple autonomous vehicles and an edge computing server and considers two main driving behaviour transition resource requests, namely car‐following behaviour requests and lane‐changing behaviour requests. Simultaneously, acknowledging that vehicles may encounter unforeseen traffic hazards when switching driving behaviours, a safety redundancy setting strategy is employed to allocate additional resources to the vehicle to ensure safety and model the vehicle resource allocation problem in the autonomous driving system. Double‐deep Q‐network (DDQN) is then used to solve this model and maximize the total system utility by comprehensively considering resource costs, system revenue, and autonomous vehicle safety. Finally, results from the simulation experiment indicate that the proposed dynamic resource allocation scheme, based on deep reinforcement learning for autonomous vehicles under edge computing, not only greatly improves the system's benefits and reduces processing delays compared to traditional greedy algorithms and value iteration, but also effectively ensures security.
- 8 February 2024
- Published by Institution of Engineering and Technology (IET) in IET Intelligent Transport Systems
- Vol. 18 (6) , 1109-1120
Open AccessJoint resource allocation and security redundancy for autonomous driving based on deep reinforcement learning algorithm
Autonomous vehicles navigating urban roads require technology that combines low latency with high computing power. The limited resources of the vehicle itself compel it to offload task requirements to edge server (ES) for processing assistance. However, as the number of vehicles continues to increase, how edge servers reasonably allocate limited resources to autonomous vehicles becomes critical to the success of urban intelligent transportation services. This paper establishes an urban road scenario with multiple autonomous vehicles and an edge computing server and considers two main driving behaviour transition resource requests, namely car‐following behaviour requests and lane‐changing behaviour requests. Simultaneously, acknowledging that vehicles may encounter unforeseen traffic hazards when switching driving behaviours, a safety redundancy setting strategy is employed to allocate additional resources to the vehicle to ensure safety and model the vehicle resource allocation problem in the autonomous driving system. Double‐deep Q‐network (DDQN) is then used to solve this model and maximize the total system utility by comprehensively considering resource costs, system revenue, and autonomous vehicle safety. Finally, results from the simulation experiment indicate that the proposed dynamic resource allocation scheme, based on deep reinforcement learning for autonomous vehicles under edge computing, not only greatly improves the system's benefits and reduces processing delays compared to traditional greedy algorithms and value iteration, but also effectively ensures security.