Encouraging Guidance: Floating Target Tracking Technology for Airborne Robotic Arm Based on Reinforcement Learning
Open Access
Abstract
Aerial robots equipped with operational robotic arms are a powerful means of achieving aerial contact operations, and their core competitiveness lies in target tracking control at the end of the airborne robotic arm (ARA). In order to improve the learning efficiency and flexibility of the ARA control algorithm, this paper proposes the encouraging guidance of an actor–critic (Eg-ac) algorithm based on the actor–critic (AC) algorithm and applies it to the floating target tracking control of ARA. It can quickly lock in the exploration direction and achieve stable tracking without increasing the learning cost. Firstly, this paper establishes approximate functions, policy functions, and encouragement functions for the state value of ARA. Secondly, an adoption rate controller (ARC) module was designed based on the concept of heavy rewards and light punishments (HRLP). Then, the kinematic and dynamic models of ARA were established. Finally, simulation was conducted using stable baselines3 (SB3). The experimental results show that, under the same computational cost, the convergence speed of the Eg-ac is improved by 21.4% compared to deep deterministic policy gradient (DDPG). Compared with soft actor–critic (SAC) and DDPG, Eg-ac has improved learning efficiency by at least 20% and has a more agile and stable floating target tracking effect.Keywords
Funding Information
- Guangxi Power Grid Company’s 2023 Science and Technology Innovation Project (GXKJXM20230169, [2020]2Y044)
- Guizhou Provincial Science and Technology Projects (GXKJXM20230169, [2020]2Y044)
This publication has 21 references indexed in Scilit:
- An obstacle avoidance method for robotic arm based on reinforcement learningIndustrial Robot: the international journal of robotics research and application, 2024
- Reinforcement learning vibration control and trajectory planning optimization of translational flexible hinged plate systemEngineering Applications of Artificial Intelligence, 2024
- A Hybrid Reinforcement Learning Approach With a Spiking Actor Network for Efficient Robotic Arm Target ReachingIEEE Robotics and Automation Letters, 2023
- Positioning of the Robotic Arm Using Different Reinforcement Learning AlgorithmsInternational Journal of Control, Automation, and Systems, 2021
- A Practical Visual Servo Control for Aerial Manipulation Using a Spherical Projection ModelIEEE Transactions on Industrial Electronics, 2019
- A Survey on Load Transportation Using Multirotor UAVsJournal of Intelligent & Robotic Systems, 2019
- Physical-Virtual Impedance Control in Ultralightweight and Compliant Dual-Arm Aerial ManipulatorsIEEE Robotics and Automation Letters, 2018
- Human-level control through deep reinforcement learningNature, 2015
- Hybrid predictive control of a coaxial aerial robot for physical interaction through contactControl Engineering Practice, 2014
- Policy Gradient Methods for Robotics2006