Category-Level Metric Scale Object Shape and Pose Estimation
- 8 September 2021
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Robotics and Automation Letters
- Vol. 6 (4) , 8575-8582
Abstract
Advances in deep learning recognition have led to accurate object detection with 2D images. However, these 2D perception methods are insufficient for complete 3D world information. Concurrently, advanced 3D shape estimation approaches focus on the shape itself, without considering metric scale. These methods cannot determine the accurate location and orientation of objects. To tackle this problem, we propose a framework that jointly estimates a metric scale shape and pose from a single RGB image. Our framework has two branches: the Metric Scale Object Shape branch (MSOS) and the Normalized Object Coordinate Space branch (NOCS). The MSOS branch estimates the metric scale shape observed in the camera coordinates. The NOCS branch predicts the normalized object coordinate space (NOCS) map and performs similarity transformation with the rendered depth map from a predicted metric scale mesh to obtain 6D pose and size. Additionally, we introduce the Normalized Object Center Estimation (NOCE) to estimate the geometrically aligned distance from the camera to the object center. We validated our method on both synthetic and real-world datasets to evaluate category-level object pose and shape.Keywords
All Related Versions
Funding Information
- Technology Innovation Program (10070171)
- Ministry of Trade, Industry and Energy
This publication has 24 references indexed in Scilit:
- Cartman: The Low-Cost Cartesian Manipulator that Won the Amazon Robotics Challenge2018
- DeformNet: Free-Form Deformation Network for 3D Shape Reconstruction from a Single Image2018
- BB8: A Scalable, Accurate, Robust to Partial Occlusion Method for Predicting the 3D Poses of Challenging Objects without Using Depth2017
- A Point Set Generation Network for 3D Object Reconstruction from a Single Image2017
- 3D-R2N2: A Unified Approach for Single and Multi-view 3D Object Reconstruction2016
- Indoor Segmentation and Support Inference from RGBD Images2012
- EPnP: An Accurate O(n) Solution to the PnP ProblemInternational Journal of Computer Vision, 2008
- Efficient variants of the ICP algorithm2002
- Least-squares estimation of transformation parameters between two point patternsIEEE Transactions on Pattern Analysis and Machine Intelligence, 1991
- Random sample consensusCommunications of the ACM, 1981