CSR-Net: Learning Adaptive Context Structure Representation for Robust Feature Correspondence
- 15 April 2022
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Image Processing
- Vol. 31 (10577149) , 3197-3210
Abstract
Feature matching, which refers to identifying and then corresponding the same or similar visual pattern from two or more images, is a key technique in any image processing task that requires establishing good correspondences between images. Given potential correspondences (matches) in two scenes, a novel whole-part deep learning framework, termed as Context Structure Representation Network (CSR-Net), is designed to infer the probabilities of arbitrary correspondences being inliers. Traditional approaches commonly build the local relation between correspondences by manually engineered criteria. Different from existing attempts, the main idea of our work is to learn explicitly neighborhood structure of each correspondence, allowing us to formulate the matching problem into a dynamic local structure consensus evaluation in an end-to-end fashion. For this purpose, we propose a permutation-invariant STructure Representation (STR) learning module, which can easily merge different types of networks into a unified architecture to deal with sparse matches directly. By the collaborative use of STR, we introduce a Context-Aware Attention (CAA) mechanism to adaptively re-calibrate structure features via a rotation-invariant context aware encoding and simple feature gating, thus arising the ability of fine-grained patterns recognition. Moreover, to further weaken the cost of establishing reliable correspondences, the CSR-Net is formulated as whole-part consensus learning, where the aim of whole level is compensating rigid transformations. In order to demonstrate our CSR-Net can effectively boost the baselines, we intensively experiment on image matching and other visual tasks. The results of the experiment confirm that the matching performances of CSR-Net have significantly improved over nine state-of-the-art competitors.Keywords
Funding Information
- National Natural Science Foundation of China (41971392)
- Graduate Research Innovation Fund of Yunnan Normal University (YJSJJ21-B73)
This publication has 39 references indexed in Scilit:
- Structure-from-Motion Revisited2016
- ORB-SLAM: A Versatile and Accurate Monocular SLAM SystemIEEE Transactions on Robotics, 2015
- Rejecting Mismatches by Correspondence FunctionInternational Journal of Computer Vision, 2010
- Object retrieval with large vocabularies and fast spatial matching2007
- MonoSLAM: Real-Time Single Camera SLAMIEEE Transactions on Pattern Analysis and Machine Intelligence, 2007
- A Comparison of Affine Region DetectorsInternational Journal of Computer Vision, 2005
- Matching with PROSAC — Progressive Sample Consensus2005
- Distinctive Image Features from Scale-Invariant KeypointsInternational Journal of Computer Vision, 2004
- Locally Optimized RANSAC2003
- Random sample consensusCommunications of the ACM, 1981