PIZZA: A Powerful Image-only Zero-Shot Zero-CAD Approach to 6 DoF Tracking
Authors:
- 1 September 2022
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 515-525
Abstract
Estimating the relative pose of a new object without prior knowledge is a hard problem, while it is an ability very much needed in robotics and Augmented Reality. We present a method for tracking the 6D motion of objects in RGB video sequences when neither the training images nor the 3D geometry of the objects are available. In contrast to previous works, our method can therefore consider unknown objects in open world instantly, without requiring any prior information or a specific training phase. We consider two architectures, one based on two frames, and the other relying on a Transformer Encoder, which can exploit an arbitrary number of past frames. We train our architectures using only synthetic renderings with domain randomization. Our results on challenging datasets are on par with previous works that require much more information (training images of the target objects, 3D models, and/or depth data). Our source code is available at https://github.com/nv-nguyen/pizza.Keywords
All Related Versions
This publication has 50 references indexed in Scilit:
- T-LESS: An RGB-D Dataset for 6D Pose Estimation of Texture-Less Objects2017
- Deep Residual Learning for Image Recognition2016
- A Dataset for Improved RGBD-Based Object Detection and Pose Estimation for Warehouse Pick-and-PlaceIEEE Robotics and Automation Letters, 2016
- A Versatile Learning-Based 3D Temporal Tracker: Scalable, Robust, Online2015
- 3D ShapeNets: A deep representation for volumetric shapes2015
- Learning descriptors for object recognition and 3D pose estimation2015
- 6-DOF Model Based Tracking via Object Coordinate Regression2015
- Learning 6D Object Pose Estimation Using 3D Object Coordinates2014
- Multimodal cue integration through Hypotheses Verification for RGB-D object recognition and 6DOF pose estimation2013
- Particle filtering on the Euclidean group: framework and applicationsRobotica, 2007