Tac2Pose: Tactile Object Pose Estimation from the First Touch
Authors:
- Maria BauzaMBMaria Bauza
- Antonia BronarsABAntonia Bronars
- Alberto RodriguezARAlberto Rodriguez
Preprint
Abstract
In this paper, we present Tac2Pose, an object-specific approach to tactile pose estimation from the first touch for known objects. Given the object geometry, we learn a tailored perception model in simulation that estimates a probability distribution over possible object poses given a tactile observation. To do so, we simulate the contact shapes that a dense set of object poses would produce on the sensor. Then, given a new contact shape obtained from the sensor, we match it against the pre-computed set using an object-specific embedding learned using contrastive learning. We obtain contact shapes from the sensor with an object-agnostic calibration step that maps RGB tactile observations to binary contact shapes. This mapping, which can be reused across object and sensor instances, is the only step trained with real sensor data. This results in a perception model that localizes objects from the first real tactile observation. Importantly, it produces pose distributions and can incorporate additional pose constraints coming from other perception systems, contacts, or priors. We provide quantitative results for 20 objects. Tac2Pose provides high accuracy pose estimations from distinctive tactile observations while regressing meaningful pose distributions to account for those contact shapes that could result from different object poses. We also test Tac2Pose on object models reconstructed from a 3D scanner, to evaluate the robustness to uncertainty in the object model. Finally, we demonstrate the advantages of Tac2Pose compared with three baseline methods for tactile pose estimation: directly regressing the object pose with a neural network, matching an observed contact to a set of possible contacts using a standard classification neural network, and direct pixel comparison of an observed contact with a set of possible contacts. Website: http://mcube.mit.edu/research/tac2pose.htmlKeywords
All Related Versions
This publication has 30 references indexed in Scilit:
- CosyPose: Consistent Multi-view Multi-object 6D Pose Estimation2020
- Momentum Contrast for Unsupervised Visual Representation Learning2020
- DeepIM: Deep Iterative Matching for 6D Pose EstimationInternational Journal of Computer Vision, 2019
- On the Continuity of Rotation Representations in Neural Networks2019
- Robotic tactile perception of object properties: A reviewMechatronics, 2017
- Deep Residual Learning for Image Recognition2016
- Localizing the object contact through matching tactile features with visual map2015
- Global estimation of an object’s pose using tactile sensingAdvanced Robotics, 2015
- A new method of constructing a grid in the space of 3D rotations and its applications to texture analysisModelling and Simulation in Materials Science and Engineering, 2014
- Object mapping, recognition, and localization from tactile geometry2011