KeyPose: Multi-View 3D Labeling and Keypoint Estimation for Transparent Objects
Authors:
- 1 June 2020
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 11599-11607
Abstract
Estimating the 3D pose of desktop objects is crucial for applications such as robotic manipulation. Many existing approaches to this problem require a depth map of the object for both training and prediction, which restricts them to opaque, lambertian objects that produce good returns in an RGBD sensor. In this paper we forgo using a depth sensor in favor of raw stereo input. We address two problems: first, we establish an easy method for capturing and labeling 3D keypoints on desktop objects with an RGB camera; and second, we develop a deep neural network, called KeyPose, that learns to accurately predict object poses using 3D keypoints, from stereo input, and works even for transparent objects. To evaluate the performance of our method, we create a dataset of 15 clear objects in five classes, with 48K 3D-keypoint labeled images. We train both instance and category models, and show generalization to new textures, poses, and objects. KeyPose surpasses state-of-the-art performance in 3D pose estimation on this dataset by factors of 1.5 to 3.5, even in cases where the competing method is provided with ground-truth depth. Stereo input is essential for this performance as it improves results compared to using monocular input by a factor of 2. We will release a public version of the data capture and labeling pipeline, the transparent object database, and the KeyPose models and evaluation code. Project website: https://sites.google.com/corp/view/keypose.Keywords
All Related Versions
This publication has 22 references indexed in Scilit:
- BB8: A Scalable, Accurate, Robust to Partial Occlusion Method for Predicting the 3D Poses of Challenging Objects without Using Depth2017
- End-to-End Learning of Geometry and Context for Deep Stereo Regression2017
- Geometric Loss Functions for Camera Pose Regression with Deep Learning2017
- AprilTag 2: Efficient and robust fiducial detection2016
- Seeing Glassware: from Edge Detection to Pose Estimation and Shape Recovery2016
- A Large Dataset to Train Convolutional Networks for Disparity, Optical Flow, and Scene Flow Estimation2016
- Pose estimation of rigid transparent objects in transparent clutter2013
- Multimodal templates for real-time detection of texture-less objects in heavily cluttered scenes2011
- Robotic Grasping of Novel Objects using VisionThe International Journal of Robotics Research, 2008
- A method for registration of 3-D shapesIEEE Transactions on Pattern Analysis and Machine Intelligence, 1992