6-DoF Pose Estimation Using an End-to-End Pipeline of Neural Network With BP-n-P
Open Access
- 3 December 2024
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Sensors Journal
- Vol. 25 (2) , 3837-3847
Abstract
This paper presents a single-shot, end-to-end deep learning architecture for object detection and 6-DoF pose estimation using an RGB image without post-processing. Previous studies have focused on pose estimation using a two-stage pipeline consisting of a feature extraction network and a pose estimation stage. However, we propose a new network structure that consolidates these two stages into a single-stage architecture to enhance the performance. Our proposed framework involves an end-to-end pipeline that integrates a YOLO architecture with a back-propagating perspective-n-point (BP-n-P) layer to predict the object’s 6-DoF pose directly from an RGB image. The model aims to predict the 2D image coordinates of the corners and center of the 3D tight bounding box around the object in the image. Here, the network back-propagates the pose estimation error through the P-n-P solver to learn the correct object pose. The LineMod dataset, which includes 15,783 RGB images and 13 different object types, was used in the experiment. In the case of single-object pose estimation, we obtained a mean pose accuracy of 71.27% and a mean pixel error of 9.16 pixels without any post-processing or pose refinement. Our approach significantly outperformed other single-shot CNN approaches without refinement conditions such as BB8, SSD-6D, YOLO-6D, PVNet, and GDR-Net. The proposed approach is suitable for robotic manipulation, self-driving cars, and augmented reality.Keywords
Funding Information
- National Science and Technology Council, Taiwan (NSTC 112-2218-E-027-006)
This publication has 34 references indexed in Scilit:
- Deep learning-based object detection in augmented reality: A systematic reviewComputers in Industry, 2022
- PVNet: Pixel-Wise Voting Network for 6DoF Pose Estimation2019
- Convolutional Neural Network-Based Real-Time Object Detection and Tracking for Parrot AR Drone 2IEEE Access, 2019
- DeepIM: Deep Iterative Matching for 6D Pose Estimation2018
- MobileNetV2: Inverted Residuals and Linear Bottlenecks2018
- Real-Time Seamless Single Shot 6D Object Pose Prediction2018
- Uncertainty-Driven 6D Pose Estimation of Objects and Scenes from a Single RGB Image2016
- You Only Look Once: Unified, Real-Time Object Detection2016
- Technical Demonstration on Model Based Training, Detection and Pose Estimation of Texture-Less 3D Objects in Heavily Cluttered Scenes2012
- The Implicit Function Theorem2003