Bio-Inspired Proprioceptive Touch of a Soft Finger with Inner-Finger Kinesthetic Perception
Open Access
- 21 October 2023
- journal article
- research article
- Published by MDPI AG in Biomimetics
- Vol. 8 (6) , 501
Abstract
In-hand object pose estimation is challenging for humans and robots due to occlusion caused by the hand and object. This paper proposes a soft finger that integrates inner vision with kinesthetic sensing to estimate object pose inspired by human fingers. The soft finger has a flexible skeleton and skin that adapts to different objects, and the skeleton deformations during interaction provide contact information obtained by the image from the inner camera. The proposed framework is an end-to-end method that uses raw images from soft fingers to estimate in-hand object pose. It consists of an encoder for kinesthetic information processing and an object pose and category estimator. The framework was tested on seven objects, achieving an impressive error of 2.02 mm and 11.34 degrees for pose error and 99.05% for classification.Keywords
Funding Information
- Ministry of Science and Technology of China (2022YFB4701200, 62206119, 52335003, JCYJ20220818100417038, ZDSYS20220527171403009, SGDX20220530110804030)
- National Natural Science Foundation of China (2022YFB4701200, 62206119, 52335003, JCYJ20220818100417038, ZDSYS20220527171403009, SGDX20220530110804030)
- Science, Technology and Innovation Commission of Shenzhen Municipality (2022YFB4701200, 62206119, 52335003, JCYJ20220818100417038, ZDSYS20220527171403009, SGDX20220530110804030)
- Guangdong Provincial Key Laboratory of Human Augmentation and Rehabilitation Robotics in Universities (2022YFB4701200, 62206119, 52335003, JCYJ20220818100417038, ZDSYS20220527171403009, SGDX20220530110804030)
This publication has 30 references indexed in Scilit:
- Fusing Joint Measurements and Visual Features for In-Hand Object Pose EstimationIEEE Robotics and Automation Letters, 2018
- Benchmarking 6DOF Outdoor Visual Localization in Changing Conditions2018
- Toward Robotic ManipulationAnnual Review of Control, Robotics, and Autonomous Systems, 2018
- Tactile-Based In-Hand Object Pose EstimationAdvances in Intelligent Systems and Computing, 2017
- GelSight: High-Resolution Robot Tactile Sensors for Estimating Geometry and ForceSensors, 2017
- Combining finger vision and optical tactile sensing: Reducing and handling errors while cutting vegetables2016
- Deep Residual Learning for Image Recognition2016
- Online in-hand object localization2013
- Tactile Sensing—From Humans to HumanoidsIEEE Transactions on Robotics, 2010
- Identifying objects by touch: An “expert system”Perception & Psychophysics, 1985