Covariant Peak Constraint for Accurate Keypoint Detection and Keypoint-Specific Descriptor Learning
- 15 November 2023
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Multimedia
- Vol. 26 (15209210) , 5383-5397
Abstract
Local feature extraction consists of keypoint detection and local descriptor extraction. Firstly, in keypoint detector learning, existing covariance constraint loss functions cannot constrain the probability distribution shapes in local probability maps that surround keypoints. And existing auxiliary peak loss functions, which are used to alleviate the problem, impair the performance of local feature methods. To solve this problem, we propose a novel Covariant Peak constraint Loss (CP Loss) which is defined as the expectations of local probability maps' position errors. Minimizing our CP Loss can make local probability maps accurately peak at reliable keypoints. Secondly, in descriptor learning, the Neural Reprojection Error (NRE) aims at constraining dense descriptor maps of images. But we argue that only those descriptors of keypoints need to be constrained. Thus, we propose a novel Conditional Neural Reprojection Error (CNRE) that is only conditioned on keypoints. Compared with the NRE, our CNRE can achieve much higher efficiency and produce more keypoint-specific descriptors with better matching performance. We use our CP Loss and CNRE to train a local feature network named as CPCN-Feat. Experimental results show that our CPCN-Feat achieves state-of-the-art performance on four challenging benchmarks.Keywords
Funding Information
- National Key Research and Development Program of China (2022YFB3303202)
- National Natural Science Foundation of China (62202468, 62002359)
This publication has 47 references indexed in Scilit:
- Reference Pose Generation for Long-term Visual Localization via Learned Features and View SynthesisInternational Journal of Computer Vision, 2020
- ShuffleNet V2: Practical Guidelines for Efficient CNN Architecture Design2018
- GeoDesc: Learning Local Descriptors by Integrating Geometry Constraints2018
- L2-Net: Deep Learning of Discriminative Patch Descriptor in Euclidean Space2017
- Comparative Evaluation of Hand-Crafted and Learned Local Features2017
- HPatches: A Benchmark and Evaluation of Handcrafted and Learned Local Descriptors2017
- Learning Discriminative and Transformation Covariant Local Feature Detectors2017
- Learning Covariant Feature Detectors2016
- Pixelwise View Selection for Unstructured Multi-View Stereo2016
- Structure-from-Motion Revisited2016