Abstract
This paper presents a single-shot, end-to-end deep learning architecture for object detection and 6-DoF pose estimation using an RGB image without post-processing. Previous studies have focused on pose estimation using a two-stage pipeline consisting of a feature extraction network and a pose estimation stage. However, we propose a new network structure that consolidates these two stages into a single-stage architecture to enhance the performance. Our proposed framework involves an end-to-end pipeline that integrates a YOLO architecture with a back-propagating perspective-n-point (BP-n-P) layer to predict the object’s 6-DoF pose directly from an RGB image. The model aims to predict the 2D image coordinates of the corners and center of the 3D tight bounding box around the object in the image. Here, the network back-propagates the pose estimation error through the P-n-P solver to learn the correct object pose. The LineMod dataset, which includes 15,783 RGB images and 13 different object types, was used in the experiment. In the case of single-object pose estimation, we obtained a mean pose accuracy of 71.27% and a mean pixel error of 9.16 pixels without any post-processing or pose refinement. Our approach significantly outperformed other single-shot CNN approaches without refinement conditions such as BB8, SSD-6D, YOLO-6D, PVNet, and GDR-Net. The proposed approach is suitable for robotic manipulation, self-driving cars, and augmented reality.
Funding Information
  • National Science and Technology Council, Taiwan (NSTC 112-2218-E-027-006)