Deep Learning-Based 3D Pose Reconstruction of an Underwater Soft Robotic Hand and Its Biomimetic Evaluation
- 10 August 2022
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Robotics and Automation Letters
- Vol. 7 (4) , 11070-11077
Abstract
Soft robotic hand shows considerable promise for various grasping applications. However, the sensing and reconstruction of the robot pose will cause limitation during the design and fabrication. In this work, we present a novel 3D pose reconstruction approach to analyze the grasping motion of a bidirectional soft robotic hand using experiment videos. The images from top, front, back, left and right views were collected using an one-camera-multiple-mirror (OCMM) imaging device. The experiment videos about underwater grasp were captured and further analyzed. The coordinate and orientation information of soft fingers are detected based on deep learning methods. A pretrained Faster RCNN model is used to detect the position of fingertips, while a pretrained U-Net classifier is applied to determine the side shape of the fingers. Based on the kinematic modelling, the corresponding coordinate and orientation databases are established. The 3D pose reconstructed results present a satisfactory performance and effectiveness. Based on efficacy coefficient method, the finger contribution of the bending angle and distance between fingertips of soft robotic hand was analyzed compared with that of human hand. The results show that the soft robot hand performs a human-like motion in both single-direction and bidirectional grasping.Keywords
Funding Information
- National Natural Science Foundation of China (51875113)
- Natural Science Joint Guidance Foundation of the Heilongjiang Province of China (LH2019E027)
- China Scholarship Council (201906680045)
This publication has 22 references indexed in Scilit:
- Machine learning-based multi-modal information perception for soft robotic handsTsinghua Science and Technology, 2020
- Quantifying Dynamic Shapes in Soft MorphologiesSoft Robotics, 2019
- Ultragentle manipulation of delicate structures using a soft robotic gripperScience Robotics, 2019
- Soft Robotic GrippersAdvanced Materials, 2018
- Soft robotics: Technologies and systems pushing the boundaries of robot abilitiesScience Robotics, 2016
- Faster R-CNN: Towards Real-Time Object Detection with Region Proposal NetworksIEEE Transactions on Pattern Analysis and Machine Intelligence, 2016
- Soft Robotic Grippers for Biological Sampling on Deep ReefsSoft Robotics, 2016
- A novel type of compliant and underactuated robotic hand for dexterous graspingThe International Journal of Robotics Research, 2015
- Vision-based 3D motion reconstruction of octopus arm swimming and comparison with an 8-arm underwater robot2015
- Design and Kinematic Modeling of Constant Curvature Continuum Robots: A ReviewThe International Journal of Robotics Research, 2010