A Concurrent Multiscale Detector for End-to-End Image Matching
- 8 August 2022
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Neural Networks and Learning Systems
- Vol. 35 (3) , 3560-3574
Abstract
This article focuses on end-to-end image matching through joint key-point detection and descriptor extraction. To find repeatable and high discrimination key points, we improve the deep matching network from the perspectives of network structure and network optimization. First, we propose a concurrent multiscale detector (CS-det) network, which consists of several parallel convolutional networks to extract multiscale features and multilevel discriminative information for key-point detection. Moreover, we introduce an attention module to fuse the response maps of various features adaptively. Importantly, we propose two novel rank consistent losses (RC-losses) for network optimization, significantly improving image matching performances. On the one hand, we propose a score rank consistent loss (RC-S-loss) to ensure that the key points have high repeatability. Different from the score difference loss merely focusing on the absolute score of an individual key point, our proposed RC-S-loss pays more attention to the relative score of key points in the image. On the other hand, we propose a score-discrimination RC-loss to ensure that the key point has high discrimination, which can reduce the confusion from other key points in subsequent matching and then further enhance the accuracy of image matching. Extensive experimental results demonstrate that the proposed CS-det improves the mean matching result of deep detector by 1.4%–2.1%, and the proposed RC-losses can boost the matching performances by 2.7%–3.4% than score difference loss. Our source codes are available at https://github.com/iquandou/CS-Net.Keywords
Funding Information
- Key Research and Development Program of Shannxi (2021ZDLGY01-06, 2022ZDLGY01-12)
- National Key Research and Development Program of China (2021ZD0110404)
- National Natural Science Foundation of China (62171347)
- Key Scientific Technological Innovation Research Project by Ministry of Education
This publication has 53 references indexed in Scilit:
- Learning Covariant Feature Detectors2016
- LIFT: Learned Invariant Feature Transform2016
- Learning Local Image Descriptors with Deep Siamese and Triplet Convolutional Networks by Minimizing Global Loss Functions2016
- Learning to Assign Orientations to Feature Points2016
- Structure-from-Motion Revisited2016
- Discriminative Learning of Deep Convolutional Feature Point Descriptors2015
- Image Matching Using Generalized Scale-Space Interest PointsJournal of Mathematical Imaging and Vision, 2014
- ASIFT: A New Framework for Fully Affine Invariant Image ComparisonSIAM Journal on Imaging Sciences, 2009
- Speeded-Up Robust Features (SURF)Computer Vision and Image Understanding, 2008
- Image registration methods: a surveyImage and Vision Computing, 2003