Fan Zhang
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Previous Organizations / Institutes
The organizations / institutes from the scholar's other articles in the past ten years
Fan Zhang has a total of 36 co-authors. Here are the top co-authors by number of shared publications:
- Guangjie Han
Hohai University
19 shared publications - Chuan Lin
Software College, Northeastern University, Shenyang, China
9 shared publications - Li Liu
School of Artificial Intelligence and Computer Science, Jiangnan University, Wuxi, China
7 shared publications Yun Hou
College of Computer Science and Software Engineering, Hohai University, Nanjing, China
7 shared publications- Yuanbo Zhu
College of Artificial Intelligence and Automation, Hohai University, Changzhou, China
4 shared publications - Yan Peng
Shanghai University
3 shared publications
Latest Publications
- 25 May 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Mobile Computing
- Vol. PP (15361233) , 1-17
Multi-AUV Cooperative Path Planning Based on Plasticity Injection TD3 Framework for IoUT Data Collection
Multi-agent reinforcement learning for multi autonomous underwater vehicle path planning faces significant technical challenges in data collection tasks within the Internet of Underwater Things. These challenges are particularly pronounced under dynamic underwater conditions, where the policy network tends to prematurely converge on suboptimal policies, thereby undermining its adaptability to environmental variations. As adaptability continues to decline during extended exploration, training stagnates, and learning efficiency decreases. To address these challenges, we propose an adaptive plasticity injection based twin delayed deep deterministic policy gradient (PITD3) framework that integrates an adaptive trigger and a dual-mode plasticity injection method. The adaptive trigger continuously monitors fluctuations in reward and temporal difference loss, activating plasticity restoration when learning stagnates or exploration becomes inefficient. The dual-mode plasticity injection method reinitializes the actor head and critic heads through selective parameter injection, restoring learning capacity and preventing long-term performance degradation. Experimental results demonstrate that the proposed PITD3 framework out performs baseline frameworks, achieving substantial improvements in collision avoidance, data collection rate, and energy efficiency.
- 25 May 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Mobile Computing
- Vol. PP (15361233) , 1-17
Multi-AUV Cooperative Path Planning Based on Plasticity Injection TD3 Framework for IoUT Data Collection
Multi-agent reinforcement learning for multi autonomous underwater vehicle path planning faces significant technical challenges in data collection tasks within the Internet of Underwater Things. These challenges are particularly pronounced under dynamic underwater conditions, where the policy network tends to prematurely converge on suboptimal policies, thereby undermining its adaptability to environmental variations. As adaptability continues to decline during extended exploration, training stagnates, and learning efficiency decreases. To address these challenges, we propose an adaptive plasticity injection based twin delayed deep deterministic policy gradient (PITD3) framework that integrates an adaptive trigger and a dual-mode plasticity injection method. The adaptive trigger continuously monitors fluctuations in reward and temporal difference loss, activating plasticity restoration when learning stagnates or exploration becomes inefficient. The dual-mode plasticity injection method reinitializes the actor head and critic heads through selective parameter injection, restoring learning capacity and preventing long-term performance degradation. Experimental results demonstrate that the proposed PITD3 framework out performs baseline frameworks, achieving substantial improvements in collision avoidance, data collection rate, and energy efficiency.
- 29 April 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Mobile Computing
- Vol. PP (15361233) , 1-17
Multi-AUV Path Planning based on Stackelberg TD3 Framework with Adaptive S-surface Control for IoUT Data Collection
Multi-autonomous underwater vehicle path planning for data collection in the internet of underwater things faces significant technical challenges. Strong ocean currents and complex terrain often cause vehicle motion to drift, leading to inaccurate paths and weak control. Existing reinforcement learning methods often struggle with training instability and cannot bridge the gap between high-level planning and low-level execution. To address these challenges, we propose a Stackelberg twin delayed deep deterministic policy gradient-based adaptive S-surface controller (STTD3-ASC). This controller integrates strategic coordination and robust control within a hierarchical architecture. Specifically, the proposed STTD3 model treats the actor-critic interaction as a two-player, general-sum Stackelberg game, with the actor as the leader and the critic as the follower. The leader optimizes its policy via an implicit gradient-based Stackelberg update that explicitly anticipates the critic's response, thereby accelerating convergence and improving training stability. Then, the high-level coordination policies generated by STTD3 are subsequently translated by the ASC model into precise low-level control signals. The hierarchical framework from STTD3 to ASC ensures global path planning along with robust local control. Experimental results show that the proposed STTD3-ASC framework outperforms the baseline in sea scenarios, achieving substantial improvements in collision avoidance, data collection rate, and energy consumption. Visualization videos of the multi-AUV coordination process can be viewed online at: https://www.youtube.com/watch?v=rVWOCm2nkHE&list=PLopigmRIskSa3zI37iiZZYvoEHrvijzia&index=1.
- 29 April 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Mobile Computing
- Vol. PP (15361233) , 1-17
Multi-AUV Path Planning based on Stackelberg TD3 Framework with Adaptive S-surface Control for IoUT Data Collection
Multi-autonomous underwater vehicle path planning for data collection in the internet of underwater things faces significant technical challenges. Strong ocean currents and complex terrain often cause vehicle motion to drift, leading to inaccurate paths and weak control. Existing reinforcement learning methods often struggle with training instability and cannot bridge the gap between high-level planning and low-level execution. To address these challenges, we propose a Stackelberg twin delayed deep deterministic policy gradient-based adaptive S-surface controller (STTD3-ASC). This controller integrates strategic coordination and robust control within a hierarchical architecture. Specifically, the proposed STTD3 model treats the actor-critic interaction as a two-player, general-sum Stackelberg game, with the actor as the leader and the critic as the follower. The leader optimizes its policy via an implicit gradient-based Stackelberg update that explicitly anticipates the critic's response, thereby accelerating convergence and improving training stability. Then, the high-level coordination policies generated by STTD3 are subsequently translated by the ASC model into precise low-level control signals. The hierarchical framework from STTD3 to ASC ensures global path planning along with robust local control. Experimental results show that the proposed STTD3-ASC framework outperforms the baseline in sea scenarios, achieving substantial improvements in collision avoidance, data collection rate, and energy consumption. Visualization videos of the multi-AUV coordination process can be viewed online at: https://www.youtube.com/watch?v=rVWOCm2nkHE&list=PLopigmRIskSa3zI37iiZZYvoEHrvijzia&index=1.
- 5 September 2025
- Published by Elsevier in Engineering Applications of Artificial Intelligence
- Vol. 161
Feature Description Attention: Channel-independent local–global fusion for multi-scale feature representation
- 5 September 2025
- Published by Elsevier in Engineering Applications of Artificial Intelligence
- Vol. 161
Feature Description Attention: Channel-independent local–global fusion for multi-scale feature representation
- 25 August 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Intelligent Transportation Systems
- Vol. 26 (10) , 15568-15581
Curiosity-Driven Distributional Soft Actor-Critic for AUV Anti-Disturbance Path Tracking
With the advancement of marine resource exploration and exploitation technologies, autonomous underwater vehicles (AUVs) have shown significant potential to perform underwater tasks, such as pipeline maintenance. However, traditional control algorithms, such as proportional-integral-derivative, sliding mode, and model predictive control, struggle to adapt to nonlinear and current-disturbed underwater environments, which impedes their accuracy and stability in path-tracking tasks. To address these challenges, this paper proposes a curiosity-driven distributional soft actor-critic framework. The framework leverages distributional soft actor-critic algorithms to control the navigation direction of the AUV and employs traditional proportional-integral control to maintain navigation speed, creating a stable, high-precision control strategy for complex underwater environments. Building on this framework, this paper further advances its capabilities through two key improvements. First, a curiosity-driven automatic entropy adjustment technique is designed to enhance the exploration of unknown states and the utilization of similar states, thus optimizing the accuracy of path tracking. Second, a dynamic prioritized experience replay mechanism is developed to prioritize learning samples based on time-differential errors and historical rewards, thereby improving learning efficiency and stability. Simulation experiments are conducted in two different underwater environments with and without ocean currents. Compared with reinforcement learning-based methods and traditional control algorithms, the proposed method has significant advantages in terms of accuracy, stability, and adaptability.
- 25 August 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Intelligent Transportation Systems
- Vol. 26 (10) , 15568-15581
Curiosity-Driven Distributional Soft Actor-Critic for AUV Anti-Disturbance Path Tracking
With the advancement of marine resource exploration and exploitation technologies, autonomous underwater vehicles (AUVs) have shown significant potential to perform underwater tasks, such as pipeline maintenance. However, traditional control algorithms, such as proportional-integral-derivative, sliding mode, and model predictive control, struggle to adapt to nonlinear and current-disturbed underwater environments, which impedes their accuracy and stability in path-tracking tasks. To address these challenges, this paper proposes a curiosity-driven distributional soft actor-critic framework. The framework leverages distributional soft actor-critic algorithms to control the navigation direction of the AUV and employs traditional proportional-integral control to maintain navigation speed, creating a stable, high-precision control strategy for complex underwater environments. Building on this framework, this paper further advances its capabilities through two key improvements. First, a curiosity-driven automatic entropy adjustment technique is designed to enhance the exploration of unknown states and the utilization of similar states, thus optimizing the accuracy of path tracking. Second, a dynamic prioritized experience replay mechanism is developed to prioritize learning samples based on time-differential errors and historical rewards, thereby improving learning efficiency and stability. Simulation experiments are conducted in two different underwater environments with and without ocean currents. Compared with reinforcement learning-based methods and traditional control algorithms, the proposed method has significant advantages in terms of accuracy, stability, and adaptability.
- 15 August 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Mobile Computing
- Vol. 25 (1) , 1089-1103
Multi-USV Coverage Path Planning Using Spatial Graph Multi-Actor-Attention-Critic Reinforcement Learning Framework With Operator Pooling
Multi-uncrewed surface vehicle coverage path planning (MCPP) presents significant challenges in large-scale aquatic environments due to dynamic ocean currents, non-euclidean spatial structures, and task load imbalance. To address these challenges, we propose the spatial graph multi-agent actor-attention-critic (SGMAAC) framework, which integrates a novel spatial graph attention network (SpGAT) for adaptive non-euclidean feature extraction and operator pooling for dynamic path optimization and task load balance. Specifically, SpGAT captures global-local topological dependencies to enhance decision-making under irregular topography, while operator pooling employs multi-step grow, deduplicate, and exchange operations to eliminate redundant paths and balance task loads across USVs. Additionally, SGMAAC introduces a multi-objective reward function that jointly optimizes coverage efficiency, collision avoidance, and energy consumption, enabling coordinated path planning in large-scale aquatic environments. Experimental results demonstrate that SGMAAC outperforms baseline methods across diverse aquatic scenarios, achieving improvements in convergence speed, makespan, task load balance, and path costs.
- 15 August 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Mobile Computing
- Vol. 25 (1) , 1089-1103
Multi-USV Coverage Path Planning Using Spatial Graph Multi-Actor-Attention-Critic Reinforcement Learning Framework With Operator Pooling
Multi-uncrewed surface vehicle coverage path planning (MCPP) presents significant challenges in large-scale aquatic environments due to dynamic ocean currents, non-euclidean spatial structures, and task load imbalance. To address these challenges, we propose the spatial graph multi-agent actor-attention-critic (SGMAAC) framework, which integrates a novel spatial graph attention network (SpGAT) for adaptive non-euclidean feature extraction and operator pooling for dynamic path optimization and task load balance. Specifically, SpGAT captures global-local topological dependencies to enhance decision-making under irregular topography, while operator pooling employs multi-step grow, deduplicate, and exchange operations to eliminate redundant paths and balance task loads across USVs. Additionally, SGMAAC introduces a multi-objective reward function that jointly optimizes coverage efficiency, collision avoidance, and energy consumption, enabling coordinated path planning in large-scale aquatic environments. Experimental results demonstrate that SGMAAC outperforms baseline methods across diverse aquatic scenarios, achieving improvements in convergence speed, makespan, task load balance, and path costs.
- 16 May 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Sensors Journal
- Vol. 25 (13) , 25948-25962
Multi-AUV Path Planning Based on Hierarchical Opponent Multiactor-Attention-Critic Reinforcement Learning Framework
The increasing exploitation of marine resources and intensifying human activities have led to significant marine pollution, highlighting the critical challenge of pollutant dispersal in aquatic environments. To address this challenge, we propose a path-planning framework based on multi-agent reinforcement learning to enable rapid deployment of detection equipment to polluted sites. Specifically, we propose a hierarchical opponent multi-actor-attention-critic framework (HOMAAC) to optimize equipment delivery path planning. In HOMAAC, the hierarchical graphical attention mechanism is employed to enhance cooperative perception and interaction among entities. Opponent modeling is innovatively employed to enhance collaboration efficiency and reduce conflicts among autonomous underwater vehicles (AUVs). This modeling approach enables each AUV to dynamically adjust its motion trajectory, thereby improving the efficiency and safety of equipment delivery and path planning. The results of experiments conducted in a multi-agent underwater logistics delivery environment validate the superior performance of the proposed framework in terms of task completion rate and time spent in addressing the path planning issue, providing valuable insights for the field of intelligent underwater delivery and marine environment monitoring.
- 16 May 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Sensors Journal
- Vol. 25 (13) , 25948-25962
Multi-AUV Path Planning Based on Hierarchical Opponent Multiactor-Attention-Critic Reinforcement Learning Framework
The increasing exploitation of marine resources and intensifying human activities have led to significant marine pollution, highlighting the critical challenge of pollutant dispersal in aquatic environments. To address this challenge, we propose a path-planning framework based on multi-agent reinforcement learning to enable rapid deployment of detection equipment to polluted sites. Specifically, we propose a hierarchical opponent multi-actor-attention-critic framework (HOMAAC) to optimize equipment delivery path planning. In HOMAAC, the hierarchical graphical attention mechanism is employed to enhance cooperative perception and interaction among entities. Opponent modeling is innovatively employed to enhance collaboration efficiency and reduce conflicts among autonomous underwater vehicles (AUVs). This modeling approach enables each AUV to dynamically adjust its motion trajectory, thereby improving the efficiency and safety of equipment delivery and path planning. The results of experiments conducted in a multi-agent underwater logistics delivery environment validate the superior performance of the proposed framework in terms of task completion rate and time spent in addressing the path planning issue, providing valuable insights for the field of intelligent underwater delivery and marine environment monitoring.
- 20 December 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Mobile Computing
- Vol. 24 (5) , 4169-4182
Underwater Multiple AUV Cooperative Target Tracking Based on Minimal Reward Participation-Embedded MARL
Recently, the rapid advancement of Multi-Agent Reinforcement Learning (MARL) has introduced a new paradigm for intelligent underwater target tracking within Autonomous Underwater Vehicle (AUV) cluster networks, enabling these networks to intelligently collaborate in target tracking. However, the limited scalability of MARL poses significant challenges to the performance of AUV cluster networks in tracking tasks. Specifically, MARL models trained on a fixed agents lose their effectiveness when the agent count changes, underscoring the critical need to enhance MARL's scalability to accommodate an arbitrary number of agents. This paper addresses the pressing issue of MARL's scalability in the context of AUV cluster network-based target tracking. Specifically, we propose an Elastic Software-Defined Multi-Agent Reinforcement Learning (ESD-MARL) architecture to enhance the scalability of AUV cluster networks. Moreover, we propose an Incremental Multi-Agent Reinforcement Learning algorithm based on Minimal Reward Participation (IMARL-MRP) that allows for the expansion of the agents without retraining. By integrating the ESD-MARL with the IMARL-MRP, we propose an elastic underwater target tracking scheme, achieving high-performance target tracking with enhanced scalability. Evaluation results demonstrate that the proposed approach effectively enhances the scalability of MARL, enabling the arbitrary expansion of the AUV cluster network, thus supporting scalable and efficient underwater target tracking.
- 20 December 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Mobile Computing
- Vol. 24 (5) , 4169-4182
Underwater Multiple AUV Cooperative Target Tracking Based on Minimal Reward Participation-Embedded MARL
Recently, the rapid advancement of Multi-Agent Reinforcement Learning (MARL) has introduced a new paradigm for intelligent underwater target tracking within Autonomous Underwater Vehicle (AUV) cluster networks, enabling these networks to intelligently collaborate in target tracking. However, the limited scalability of MARL poses significant challenges to the performance of AUV cluster networks in tracking tasks. Specifically, MARL models trained on a fixed agents lose their effectiveness when the agent count changes, underscoring the critical need to enhance MARL's scalability to accommodate an arbitrary number of agents. This paper addresses the pressing issue of MARL's scalability in the context of AUV cluster network-based target tracking. Specifically, we propose an Elastic Software-Defined Multi-Agent Reinforcement Learning (ESD-MARL) architecture to enhance the scalability of AUV cluster networks. Moreover, we propose an Incremental Multi-Agent Reinforcement Learning algorithm based on Minimal Reward Participation (IMARL-MRP) that allows for the expansion of the agents without retraining. By integrating the ESD-MARL with the IMARL-MRP, we propose an elastic underwater target tracking scheme, achieving high-performance target tracking with enhanced scalability. Evaluation results demonstrate that the proposed approach effectively enhances the scalability of MARL, enabling the arbitrary expansion of the AUV cluster network, thus supporting scalable and efficient underwater target tracking.
- 27 November 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Internet of Things Journal
- Vol. 12 (7) , 9218-9237
Secure Data Offloading and Resource Allocation Against Hybrid Intrusions for IIoT: A Fully Decentralized Framework
Edge computing is fundamental to filling the various quality-of-service needs for Industrial Internet of Things (IIoT) applications. However, introducing edge computing to IIoT inevitably results in hybrid intrusion problems and fails to satisfy the security demands of IIoT. Fortunately, Lagrange coded computing has emerged as a low-complexity and low-overhead solution for resisting hybrid intrusions during data offloading and processing. However, how to make decentralized, accurate, and real-time encoding/offloading/decoding decisions remains challenging. This article designs a fully decentralized training and decision-making framework to address the joint secure data offloading and resource allocation problem against hybrid intrusions for dynamic and uncertain IIoT, attempting to minimize the long-run energy and delay costs while improving the data confidentiality, integrity, and availability. It is proposed a fully decentralized multiagent actor–critic-based secure data offloading (FM-SDO) algorithm to solve the secure data offloading subproblem, wherein each industrial end device utilizes its local information to learn and execute its policy independently. This algorithm improves the structure of actor and critic networks and designs a multiagent alternant updating mechanism to increase learning accuracy, convergence, and stability. Based on the received offloading decisions of each device, each edge server leverages the Lagrange multiplier approach and Karush–Kuhn–Tucker condition to make fast and decentralized resource allocation decisions. Finally, we employed an IIoT intelligent production line platform named iCandyBox to test the performance of the FM-SDO algorithm. Experiment results suggest that the FM-SDO algorithm effectively reduces the total energy and delay costs while increasing the capability of resisting hybrid intrusions.
- 27 November 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Internet of Things Journal
- Vol. 12 (7) , 9218-9237
Secure Data Offloading and Resource Allocation Against Hybrid Intrusions for IIoT: A Fully Decentralized Framework
Edge computing is fundamental to filling the various quality-of-service needs for Industrial Internet of Things (IIoT) applications. However, introducing edge computing to IIoT inevitably results in hybrid intrusion problems and fails to satisfy the security demands of IIoT. Fortunately, Lagrange coded computing has emerged as a low-complexity and low-overhead solution for resisting hybrid intrusions during data offloading and processing. However, how to make decentralized, accurate, and real-time encoding/offloading/decoding decisions remains challenging. This article designs a fully decentralized training and decision-making framework to address the joint secure data offloading and resource allocation problem against hybrid intrusions for dynamic and uncertain IIoT, attempting to minimize the long-run energy and delay costs while improving the data confidentiality, integrity, and availability. It is proposed a fully decentralized multiagent actor–critic-based secure data offloading (FM-SDO) algorithm to solve the secure data offloading subproblem, wherein each industrial end device utilizes its local information to learn and execute its policy independently. This algorithm improves the structure of actor and critic networks and designs a multiagent alternant updating mechanism to increase learning accuracy, convergence, and stability. Based on the received offloading decisions of each device, each edge server leverages the Lagrange multiplier approach and Karush–Kuhn–Tucker condition to make fast and decentralized resource allocation decisions. Finally, we employed an IIoT intelligent production line platform named iCandyBox to test the performance of the FM-SDO algorithm. Experiment results suggest that the FM-SDO algorithm effectively reduces the total energy and delay costs while increasing the capability of resisting hybrid intrusions.
- 25 September 2024
- Published by Elsevier in Computers & Security
- Vol. 148
An explainable unsupervised anomaly detection framework for Industrial Internet of Things
- 25 September 2024
- Published by Elsevier in Computers & Security
- Vol. 148
An explainable unsupervised anomaly detection framework for Industrial Internet of Things