ZWZhiyong Wang

Zhiyong Wang

*This is an algorithmically generated author record based on publicly available information.

Current Organizations / Institutes

The organizations/institutes from the scholar's latest article


Zhiyong Wang has a total of 9 co-authors. Here are the top co-authors by number of shared publications:


Latest Publications

  • Open Access

    Encouraging Guidance: Floating Target Tracking Technology for Airborne Robotic Arm Based on Reinforcement Learning

    by Jiying Wu, Zhong Yang, Haoze Zhuo, Changliang Xu, Luwei Liao, Danguo Cheng, Zhiyong Wang

    Aerial robots equipped with operational robotic arms are a powerful means of achieving aerial contact operations, and their core competitiveness lies in target tracking control at the end of the airborne robotic arm (ARA). In order to improve the learning efficiency and flexibility of the ARA control algorithm, this paper proposes the encouraging guidance of an actor–critic (Eg-ac) algorithm based on the actor–critic (AC) algorithm and applies it to the floating target tracking control of ARA. It can quickly lock in the exploration direction and achieve stable tracking without increasing the learning cost. Firstly, this paper establishes approximate functions, policy functions, and encouragement functions for the state value of ARA. Secondly, an adoption rate controller (ARC) module was designed based on the concept of heavy rewards and light punishments (HRLP). Then, the kinematic and dynamic models of ARA were established. Finally, simulation was conducted using stable baselines3 (SB3). The experimental results show that, under the same computational cost, the convergence speed of the Eg-ac is improved by 21.4% compared to deep deterministic policy gradient (DDPG). Compared with soft actor–critic (SAC) and DDPG, Eg-ac has improved learning efficiency by at least 20% and has a more agile and stable floating target tracking effect.


  • Open Access

    A Supervised Reinforcement Learning Algorithm for Controlling Drone Hovering

    by Jiying Wu, Zhong Yang, Haoze Zhuo, Changliang Xu, Chi Zhang, Naifeng He, Luwei Liao, Zhiyong Wang

    The application of drones carrying different devices for aerial hovering operations is becoming increasingly widespread, but currently there is very little research relying on reinforcement learning methods for hovering control, and it has not been implemented on physical machines. Drone’s behavior space regarding hover control is continuous and large-scale, making it difficult for basic algorithms and value-based reinforcement learning (RL) algorithms to have good results. In response to this issue, this article applies a watcher-actor-critic (WAC) algorithm to the drone’s hover control, which can quickly lock the exploration direction and achieve high robustness of the drone’s hover control while improving learning efficiency and reducing learning costs. This article first utilizes the actor-critic algorithm based on behavioral value Q (QAC) and the deep deterministic policy gradient algorithm (DDPG) for drone hover control learning. Subsequently, an actor-critic algorithm with an added watcher is proposed, in which the watcher uses a PID controller with parameters provided by a neural network as the dynamic monitor, transforming the learning process into supervised learning. Finally, this article uses a classic reinforcement learning environment library, Gym, and a current mainstream reinforcement learning framework, PARL, for simulation, and deploys the algorithm to a practical environment. A multi-sensor fusion strategy-based autonomous localization method for unmanned aerial vehicles is used for practical exercises. The simulation and experimental results show that the training episodes of WAC are reduced by 20% compared to the DDPG and 55% compared to the QAC, and the proposed algorithm has a higher learning efficiency, faster convergence speed, and smoother hovering effect compared to the QAC and DDPG.


  • Open Access

    A State-Compensated Deep Deterministic Policy Gradient Algorithm for UAV Trajectory Tracking

    by Jiying Wu, Zhong Yang, Luwei Liao, Naifeng He, Zhiyong Wang, Can Wang

    The unmanned aerial vehicle (UAV) trajectory tracking control algorithm based on deep reinforcement learning is generally inefficient for training in an unknown environment, and the convergence is unstable. Aiming at this situation, a Markov decision process (MDP) model for UAV trajectory tracking is established, and a state-compensated deep deterministic policy gradient (CDDPG) algorithm is proposed. An additional neural network (C-Net) whose input is compensation state and output is compensation action is added to the network model of a deep deterministic policy gradient (DDPG) algorithm to assist in network exploration training. It combined the action output of the DDPG network with compensated output of the C-Net as the output action to interact with the environment, enabling the UAV to rapidly track dynamic targets in the most accurate continuous and smooth way possible. In addition, random noise is added on the basis of the generated behavior to realize a certain range of exploration and make the action value estimation more accurate. The OpenAI Gym tool is used to verify the proposed method, and the simulation results show that: (1) The proposed method can significantly improve the training efficiency by adding a compensation network and effectively improve the accuracy and convergence stability; (2) Under the same computer configuration, the computational cost of the proposed algorithm is basically the same as that of the QAC algorithm (Actor-critic algorithm based on behavioral value Q) and the DDPG algorithm; (3) During the training process, with the same tracking accuracy, the learning efficiency is about 70% higher than that of QAC and DDPG; (4) During the simulation tracking experiment, under the same training time, the tracking error of the proposed method after stabilization is about 50% lower than that of QAC and DDPG.