PanoPoint: Self-Supervised Feature Points Detection and Description for 360° Panorama
Authors:
- 1 June 2023
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 6449-6458
Abstract
We introduce PanoPoint, the joint feature point detection and description applied to the nonlinear distortions and the multi-view geometry problems between 360° panoramas. Our fully convolutional model operates directly in panoramas and computes pixel-level feature point locations and associated descriptors in a single forward pass rather than performing image preprocessing (e.g. panorama to Cubemap) followed by feature detection and description. To train the PanoPoint model, we propose PanoMotion, which simulates the representation between different viewpoints and generates warped panoramas. Moreover, we propose PanoMotion Adaptation, a multi-viewpoint adaptation annotation approach for boosting feature point detection repeatability instead of manual labelling. We train on the annotated synthetic dataset generated by the above method, which outperforms the traditional and other learned approaches and achieves state-of-the-art results on repeatability, localization accuracy, point correspondence precision and real-time metrics, especially for panoramas with significant viewpoint and illumination changes.Keywords
This publication has 38 references indexed in Scilit:
- Structure-from-Motion Revisited2016
- Robot Guidance Using Machine Vision Techniques in Industrial Environments: A Comparative ReviewSensors, 2016
- Homography Estimation Between Omnidirectional Cameras Without Point Correspondences2015
- SPHORB: A Fast and Robust Binary Feature on the SphereInternational Journal of Computer Vision, 2014
- SUN3D: A Database of Big Spaces Reconstructed Using SfM and Object Labels2013
- The vSLAM Algorithm for Robust Localization and Mapping2006
- A performance evaluation of local descriptorsIEEE Transactions on Pattern Analysis and Machine Intelligence, 2005
- Distinctive Image Features from Scale-Invariant KeypointsInternational Journal of Computer Vision, 2004
- An Affine Invariant Interest Point Detector2002
- Bilinear interpolation of digital imagesUltramicroscopy, 1981