PanoPose: Self-supervised Relative Pose Estimation for Panoramic Images
Authors:
- 16 June 2024
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 20009-20018
Abstract
Scaled relative pose estimation, i.e., estimating relative rotation and scaled relative translation between two images, has always been a major challenge in global Structure-from-Motion (SfM). This difficulty arises because the two-view relative translation computed by traditional geometric vision methods, e.g. the five-point algorithm, is scaleless. Many researchers have proposed diverse translation averaging methods to solve this problem. Instead of solving the problem in the motion averaging phase, we focus on estimating scaled relative pose with the help of panoramic cameras and deep neural networks. In this paper, a novel network, namely PanoPose, is proposed to estimate the relative motion in a fully self-supervised manner and a global SfM pipeline is built for panorama images. The proposed PanoPose comprises a depth-net and a pose-net, with self-supervision achieved by reconstructing the reference image from its neighboring images based on the estimated depth and relative pose. To maintain precise pose estimation under large viewing angle differences, we randomly rotate the panoramic images and pre-train the posenet with images before and after the rotation. To enhance scale accuracy, a fusion block is introduced to incorporate depth information into pose estimation. Extensive experiments on panoramic SfM datasets demonstrate the effectiveness of PanoPose compared with state-of-the-arts.Keywords
Funding Information
- National Natural Science Foundation of China (U22B2055,U23A20386,62273345,62073320)
This publication has 40 references indexed in Scilit:
- Capturing, Reconstructing, and Simulating: The UrbanScene3D Dataset2022
- CoVisPose: Co-visibility Pose Transformer for Wide-Baseline Relative Pose Estimation in 360$^\circ $ Indoor Panoramas2022
- Correspondence Reweighted Translation Averaging2022
- PanoFormer: Panorama Transformer for Indoor 360$^{\circ }$ Depth Estimation2022
- Deep Regression on Manifolds: A 3D Rotation Case Study2021
- PanoDepth: A Two-Stage Approach for Monocular Omnidirectional Depth Estimation2021
- Tracks selection for robust, efficient and scalable large-scale structure from motionPattern Recognition, 2017
- Batched Incremental Structure-from-Motion2017
- Robust Global Translations with 1DSfM2014
- Combining two-view constraints for motion estimation2005