A Novel Deep Reinforcement Learning-Based Path/Force Cooperative Regulation Framework for Dual-Arm Object Transportation
- 22 May 2025
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Automation Science and Engineering
- Vol. 22 (15455955) , 15792-15804
Abstract
Dual-arm robots, with their high flexibility and broad operational range, are widely used in industrial and household transportation tasks. However, their high degrees of motion freedom and the closed-chain constraints formed during transportation pose challenges for path planning and force control. This study proposes a dual-agent control framework based on the Soft-Actor-Critic (SAC) algorithm of Deep Reinforcement Learning (DRL), where one agent is responsible for path planning and the other handles force control. This framework enables dual-arm robots to achieve dynamic obstacle avoidance, avoid unsolvable configurations and singularities, and generate efficient and smooth paths, while also fulfilling internal force tracking requirements based on task demands. Additionally, it addresses the gap in transferring the force control agent from simulation to real-world applications through a state mapping network, and the force control agent does not require retraining for different objects. Finally, the effectiveness of the proposed framework is validated through two scenes, including multiple bookshelves stacking and dynamic obstacle avoidance during the box transportation. Validation was also carried out with objects of different geometries and weights. Note to Practitioners—The motivation of this study is the path planning and force tracking control problem for dual-arm transportation. Existing methods rarely provide a comprehensive framework to address the task of dual-arm transporting different objects in dynamic environments. A DRL-based dual-agent control framework is proposed in this study, which simultaneously achieves task-compliant path planning and force tracking control. The trained agents exhibit strong generalization capabilities and stability, allowing them to handle objects of different geometries, weights, and stiffness without retraining. The limitation lies in the lack of methods for generating dual-arm initial grasping actions for complex-shaped objects. Future research will integrate image recognition techniques to enable grasping action generation for objects in diverse poses, further enhancing the applicability of the proposed framework.Keywords
Funding Information
- National Natural Science Foundation of China (U24A20130, 52205521, 52188102, 52090054)
This publication has 47 references indexed in Scilit:
- Learning compliant dynamical system from human demonstrations for stable force control in unknown environmentsRobotics and Computer-Integrated Manufacturing, 2023
- A systematic review on cooperative dual-arm manipulators: modeling, planning, control, and vision strategiesInternational Journal of Intelligent Robotics and Applications, 2023
- Adaptive neural synchronized impedance control for cooperative manipulators processing under uncertain environmentsRobotics and Computer-Integrated Manufacturing, 2022
- Deep learning-based optimization for motion planning of dual-arm assembly robotsComputers & Industrial Engineering, 2021
- Constrained motion planning of free-float dual-arm space manipulator via deep reinforcement learningAerospace Science and Technology, 2021
- A review On reinforcement learning: Introduction and applications in industrial process controlComputers & Chemical Engineering, 2020
- Symmetrical adaptive variable admittance control for position/force tracking of dual-arm cooperative manipulators with unknown trajectory deviationsRobotics and Computer-Integrated Manufacturing, 2019
- Kinematics Analysis and Trajectory Planning of collaborative welding robot with multiple manipulatorsProcedia CIRP, 2019
- Collision and Singularity Avoidance Path Planning of 6-DOF Dual-Arm Manipulator2018
- On the adaptive control of cooperative robots with time‐variant holonomic constraintsInternational Journal of Adaptive Control and Signal Processing, 2017