A Reinforcement Learning Method for Motion Control With Constraints on an HPN Arm
- 5 August 2022
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Robotics and Automation Letters
- Vol. 7 (4) , 12006-12013
Abstract
Soft robotic arms have shown great potential toward applications to human daily lives, which is mainly due to their infinite passive degrees of freedom and intrinsic safety. There are tasks in lives that require the motion of the robot to meet some certain pose constraints that have not been implemented through the soft arm, like delivering a glass of water. Because the workspace of the soft arm is affected by the loads or interaction, it is difficult to implement this task through the motion planning method. In this letter, we propose a Q-learning based approach to address the problem, directly achieving motion control with constraints under loads and interaction without planning. We first generate a controller for the soft arm based on Q-learning, which can operate the arm while satisfying the pose constraints when the arm is neither loaded nor interacted with the environment. Then, we introduce a process that adjusts corresponding Q values in the controller, which allows the controller to operate the arm with an unknown load or interaction while still satisfying the pose constraints. We implement the approach on our soft arm, i.e., the Honeycomb Pneumatic Network (HPN) Arm. The experiments show that the approach is effective, even when the arm reached an untrained situation or even beyond the workspace under the interaction.Keywords
Funding Information
- Open Project Fund from Shenzhen Institute of Artificial Intelligence and Robotics for Society (AC01202005010)
- National Natural Science Foundation of China (61573333)
- CAAI-Huawei MindSpore Open Fund. And it is partially supported by the 2030 National Key AI Program of China (2018AAA0100500)
- Guangdong Province R&D Program (2020B0909050001)
- Anhui Province Development and Reform Commission 2020 New Energy Vehicle Industry Innovation Development Project
- 2021 New Energy and Intelligent Connected Vehicle Innovation Project, National Natural Science Foundation of China (92048301)
- Shenzhen Yijiahe Technology R&D Co., Ltd.
This publication has 17 references indexed in Scilit:
- Control Strategies for Soft Robotic Manipulators: A SurveySoft Robotics, 2018
- Nonparametric Online Learning Control for Soft Continuum Robot: An Enabling Technique for Effective Endoscopic NavigationSoft Robotics, 2017
- Learning Closed Loop Kinematic Controllers for Continuum Manipulators in Unstructured EnvironmentsSoft Robotics, 2017
- Model-less feedback control for soft manipulators2017
- Model-free control for soft manipulators based on reinforcement learning2017
- Multiobjective Optimization for Stiffness and Position Control in a Soft Robot Arm ModuleIEEE Robotics and Automation Letters, 2017
- A two-level approach for solving the inverse kinematics of an extensible soft arm considering viscoelastic behavior2017
- Model-Less Hybrid Position/Force Control: A Minimalist Approach for Continuum Manipulators in Unknown, Constrained EnvironmentsIEEE Robotics and Automation Letters, 2016
- Design, fabrication and control of soft robotsNature, 2015
- Design and control of a soft and continuously deformable 2D robotic manipulation system2014