FZFan Zhang

Fan Zhang

*This is an algorithmically generated author record based on publicly available information.

Current Organizations / Institutes

The organizations/institutes from the scholar's latest article

Previous Organizations / Institutes

The organizations / institutes from the scholar's other articles in the past ten years


Fan Zhang has a total of 36 co-authors. Here are the top co-authors by number of shared publications:


Latest Publications

  • Multi-AUV Cooperative Path Planning Based on Plasticity Injection TD3 Framework for IoUT Data Collection

    by Yuanbo Zhu, Guangjie Han, Chuan Lin, Fan Zhang, Yun Hou

    Multi-agent reinforcement learning for multi autonomous underwater vehicle path planning faces significant technical challenges in data collection tasks within the Internet of Underwater Things. These challenges are particularly pronounced under dynamic underwater conditions, where the policy network tends to prematurely converge on suboptimal policies, thereby undermining its adaptability to environmental variations. As adaptability continues to decline during extended exploration, training stagnates, and learning efficiency decreases. To address these challenges, we propose an adaptive plasticity injection based twin delayed deep deterministic policy gradient (PITD3) framework that integrates an adaptive trigger and a dual-mode plasticity injection method. The adaptive trigger continuously monitors fluctuations in reward and temporal difference loss, activating plasticity restoration when learning stagnates or exploration becomes inefficient. The dual-mode plasticity injection method reinitializes the actor head and critic heads through selective parameter injection, restoring learning capacity and preventing long-term performance degradation. Experimental results demonstrate that the proposed PITD3 framework out performs baseline frameworks, achieving substantial improvements in collision avoidance, data collection rate, and energy efficiency.

  • Multi-AUV Path Planning based on Stackelberg TD3 Framework with Adaptive S-surface Control for IoUT Data Collection

    by Yuanbo Zhu, Guangjie Han, Chuan Lin, Fan Zhang, Yun Hou

    Multi-autonomous underwater vehicle path planning for data collection in the internet of underwater things faces significant technical challenges. Strong ocean currents and complex terrain often cause vehicle motion to drift, leading to inaccurate paths and weak control. Existing reinforcement learning methods often struggle with training instability and cannot bridge the gap between high-level planning and low-level execution. To address these challenges, we propose a Stackelberg twin delayed deep deterministic policy gradient-based adaptive S-surface controller (STTD3-ASC). This controller integrates strategic coordination and robust control within a hierarchical architecture. Specifically, the proposed STTD3 model treats the actor-critic interaction as a two-player, general-sum Stackelberg game, with the actor as the leader and the critic as the follower. The leader optimizes its policy via an implicit gradient-based Stackelberg update that explicitly anticipates the critic's response, thereby accelerating convergence and improving training stability. Then, the high-level coordination policies generated by STTD3 are subsequently translated by the ASC model into precise low-level control signals. The hierarchical framework from STTD3 to ASC ensures global path planning along with robust local control. Experimental results show that the proposed STTD3-ASC framework outperforms the baseline in sea scenarios, achieving substantial improvements in collision avoidance, data collection rate, and energy consumption. Visualization videos of the multi-AUV coordination process can be viewed online at: https://www.youtube.com/watch?v=rVWOCm2nkHE&list=PLopigmRIskSa3zI37iiZZYvoEHrvijzia&index=1.

  • Curiosity-Driven Distributional Soft Actor-Critic for AUV Anti-Disturbance Path Tracking

    by Yun Hou, Guangjie Han, Fan Zhang, Chuan Lin, Yunpeng Ma

    With the advancement of marine resource exploration and exploitation technologies, autonomous underwater vehicles (AUVs) have shown significant potential to perform underwater tasks, such as pipeline maintenance. However, traditional control algorithms, such as proportional-integral-derivative, sliding mode, and model predictive control, struggle to adapt to nonlinear and current-disturbed underwater environments, which impedes their accuracy and stability in path-tracking tasks. To address these challenges, this paper proposes a curiosity-driven distributional soft actor-critic framework. The framework leverages distributional soft actor-critic algorithms to control the navigation direction of the AUV and employs traditional proportional-integral control to maintain navigation speed, creating a stable, high-precision control strategy for complex underwater environments. Building on this framework, this paper further advances its capabilities through two key improvements. First, a curiosity-driven automatic entropy adjustment technique is designed to enhance the exploration of unknown states and the utilization of similar states, thus optimizing the accuracy of path tracking. Second, a dynamic prioritized experience replay mechanism is developed to prioritize learning samples based on time-differential errors and historical rewards, thereby improving learning efficiency and stability. Simulation experiments are conducted in two different underwater environments with and without ocean currents. Compared with reinforcement learning-based methods and traditional control algorithms, the proposed method has significant advantages in terms of accuracy, stability, and adaptability.


  • Multi-USV Coverage Path Planning Using Spatial Graph Multi-Actor-Attention-Critic Reinforcement Learning Framework With Operator Pooling

    by Yuanbo Zhu, Guangjie Han, Chuan Lin, Fan Zhang, Yun Hou

    Multi-uncrewed surface vehicle coverage path planning (MCPP) presents significant challenges in large-scale aquatic environments due to dynamic ocean currents, non-euclidean spatial structures, and task load imbalance. To address these challenges, we propose the spatial graph multi-agent actor-attention-critic (SGMAAC) framework, which integrates a novel spatial graph attention network (SpGAT) for adaptive non-euclidean feature extraction and operator pooling for dynamic path optimization and task load balance. Specifically, SpGAT captures global-local topological dependencies to enhance decision-making under irregular topography, while operator pooling employs multi-step grow, deduplicate, and exchange operations to eliminate redundant paths and balance task loads across USVs. Additionally, SGMAAC introduces a multi-objective reward function that jointly optimizes coverage efficiency, collision avoidance, and energy consumption, enabling coordinated path planning in large-scale aquatic environments. Experimental results demonstrate that SGMAAC outperforms baseline methods across diverse aquatic scenarios, achieving improvements in convergence speed, makespan, task load balance, and path costs.


  • Multi-AUV Path Planning Based on Hierarchical Opponent Multiactor-Attention-Critic Reinforcement Learning Framework

    by Yuanbo Zhu, Guangjie Han, Chuan Lin, Fan Zhang, Yun Hou

    The increasing exploitation of marine resources and intensifying human activities have led to significant marine pollution, highlighting the critical challenge of pollutant dispersal in aquatic environments. To address this challenge, we propose a path-planning framework based on multi-agent reinforcement learning to enable rapid deployment of detection equipment to polluted sites. Specifically, we propose a hierarchical opponent multi-actor-attention-critic framework (HOMAAC) to optimize equipment delivery path planning. In HOMAAC, the hierarchical graphical attention mechanism is employed to enhance cooperative perception and interaction among entities. Opponent modeling is innovatively employed to enhance collaboration efficiency and reduce conflicts among autonomous underwater vehicles (AUVs). This modeling approach enables each AUV to dynamically adjust its motion trajectory, thereby improving the efficiency and safety of equipment delivery and path planning. The results of experiments conducted in a multi-agent underwater logistics delivery environment validate the superior performance of the proposed framework in terms of task completion rate and time spent in addressing the path planning issue, providing valuable insights for the field of intelligent underwater delivery and marine environment monitoring.


  • Underwater Multiple AUV Cooperative Target Tracking Based on Minimal Reward Participation-Embedded MARL

    by Shengchao Zhu, Guangjie Han, Chuan Lin, Fan Zhang

    Recently, the rapid advancement of Multi-Agent Reinforcement Learning (MARL) has introduced a new paradigm for intelligent underwater target tracking within Autonomous Underwater Vehicle (AUV) cluster networks, enabling these networks to intelligently collaborate in target tracking. However, the limited scalability of MARL poses significant challenges to the performance of AUV cluster networks in tracking tasks. Specifically, MARL models trained on a fixed agents lose their effectiveness when the agent count changes, underscoring the critical need to enhance MARL's scalability to accommodate an arbitrary number of agents. This paper addresses the pressing issue of MARL's scalability in the context of AUV cluster network-based target tracking. Specifically, we propose an Elastic Software-Defined Multi-Agent Reinforcement Learning (ESD-MARL) architecture to enhance the scalability of AUV cluster networks. Moreover, we propose an Incremental Multi-Agent Reinforcement Learning algorithm based on Minimal Reward Participation (IMARL-MRP) that allows for the expansion of the agents without retraining. By integrating the ESD-MARL with the IMARL-MRP, we propose an elastic underwater target tracking scheme, achieving high-performance target tracking with enhanced scalability. Evaluation results demonstrate that the proposed approach effectively enhances the scalability of MARL, enabling the arbitrary expansion of the AUV cluster network, thus supporting scalable and efficient underwater target tracking.


  • Secure Data Offloading and Resource Allocation Against Hybrid Intrusions for IIoT: A Fully Decentralized Framework

    by Fan Zhang, Guangjie Han, Li Liu, Jinfang Jiang, Aohan Li, Shengchao Zhu

    Edge computing is fundamental to filling the various quality-of-service needs for Industrial Internet of Things (IIoT) applications. However, introducing edge computing to IIoT inevitably results in hybrid intrusion problems and fails to satisfy the security demands of IIoT. Fortunately, Lagrange coded computing has emerged as a low-complexity and low-overhead solution for resisting hybrid intrusions during data offloading and processing. However, how to make decentralized, accurate, and real-time encoding/offloading/decoding decisions remains challenging. This article designs a fully decentralized training and decision-making framework to address the joint secure data offloading and resource allocation problem against hybrid intrusions for dynamic and uncertain IIoT, attempting to minimize the long-run energy and delay costs while improving the data confidentiality, integrity, and availability. It is proposed a fully decentralized multiagent actor–critic-based secure data offloading (FM-SDO) algorithm to solve the secure data offloading subproblem, wherein each industrial end device utilizes its local information to learn and execute its policy independently. This algorithm improves the structure of actor and critic networks and designs a multiagent alternant updating mechanism to increase learning accuracy, convergence, and stability. Based on the received offloading decisions of each device, each edge server leverages the Lagrange multiplier approach and Karush–Kuhn–Tucker condition to make fast and decentralized resource allocation decisions. Finally, we employed an IIoT intelligent production line platform named iCandyBox to test the performance of the FM-SDO algorithm. Experiment results suggest that the FM-SDO algorithm effectively reduces the total energy and delay costs while increasing the capability of resisting hybrid intrusions.