Understanding the Limitations of CNN-Based Absolute Camera Pose Regression
Authors:
- Laura Leal-TaixeLLLaura Leal-Taixe
Top Cited Papers
- 1 June 2019
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 3297-3307
Abstract
Visual localization is the task of accurate camera pose estimation in a known scene. It is a key problem in computer vision and robotics, with applications including self-driving cars, Structure-from-Motion, SLAM, and Mixed Reality. Traditionally, the localization problem has been tackled using 3D geometry. Recently, end-to-end approaches based on convolutional neural networks have become popular. These methods learn to directly regress the camera pose from an input image. However, they do not achieve the same level of pose accuracy as 3D structure-based methods. To understand this behavior, we develop a theoretical model for camera pose regression. We use our model to predict failure cases for pose regression techniques and verify our predictions through experiments. We furthermore use our model to show that pose regression is more closely related to pose approximation via image retrieval than to accurate pose estimation via 3D structure. A key result is that current approaches do not consistently outperform a handcrafted image retrieval baseline. This clearly shows that additional research is needed before pose regression algorithms are ready to compete with structure-based methods.Keywords
All Related Versions
This publication has 61 references indexed in Scilit:
- Real-time monocular image-based 6-DoF localizationThe International Journal of Robotics Research, 2015
- Multi-output Learning for Camera Relocalization2014
- Real-Time Solution to the Absolute Pose Problem with Unknown Radial Distortion and Focal Length2013
- Graph-Based Discriminative Learning for Location Recognition2013
- Scene Coordinate Regression Forests for Camera Relocalization in RGB-D Images2013
- Accurate Image Localization Based on Google Maps Street View2010
- Image Based Localization in Urban Environments2006
- Distinctive Image Features from Scale-Invariant KeypointsInternational Journal of Computer Vision, 2004
- Video Google: a text retrieval approach to object matching in videos2003
- Random sample consensusCommunications of the ACM, 1981