In-Hand Pose Estimation Using Hand-Mounted RGB Cameras and Visuotactile Sensors
Open Access
- 13 February 2023
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 11, 17218-17232
Abstract
This paper proposes a method to estimate the 6D pose of an object grasped by a robot hand using RGB cameras mounted on the palm and visuotactile sensors installed at the fingertips. It can deal with objects made from a wide range of materials thanks to combining the two types of sensors. The method allows a robot to robot to perform in-hand pose estimation while holding the object, eliminating the need for preparatory actions or particular environmental backgrounds. The mechanism at the back of the method includes deep-learning-based background subtraction and denoising auto-encoder-based sensor fusion. With the poses estimated using the proposed method, a robot controller can rectify the grasping uncertainty and adjust the robot motion to move an object toward required goals with satisfying accuracy. We conduct various studies and analyses in the experimental section to understand the proposed method’s advantages and disadvantages. The results demonstrate the benefits of the proposed combination and mechanism. They also provide essential knowledge to readers considering using a similar configuration for estimating object poses.Keywords
This publication has 51 references indexed in Scilit:
- GelSight: High-Resolution Robot Tactile Sensors for Estimating Geometry and ForceSensors, 2017
- Tracking objects with point clouds from vision and touch2017
- Combining finger vision and optical tactile sensing: Reducing and handling errors while cutting vegetables2016
- In-Hand Object Pose Estimation Using Covariance-Based Tactile To Geometry MatchingIEEE Robotics and Automation Letters, 2016
- Localization and manipulation of small parts using GelSight tactile sensing2014
- A highly sensitive 3D-shaped tactile sensor2013
- Fast object localization and pose estimation in heavy clutter for robotic bin pickingThe International Journal of Robotics Research, 2012
- Fusion of stereo vision, force-torque, and joint sensors for estimation of in-hand object location2011
- Extracting and composing robust features with denoising autoencoders2008
- Reducing the Dimensionality of Data with Neural NetworksScience, 2006