Match Any Keypoints
- 1 January 2026
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Image Processing
- Vol. 35, 4051-4063
Abstract
Previous research on sparse feature matching typically involves a staged optimization process of keypoint detection, description, and matching. While it allows the network to adapt to specific inputs, it may limit the network's expressive capability and the overall architectural flexibility. In this study, we rethink the matching framework and propose to directly match any given keypoints, optimizing the matching network in an approximately end-to-end manner. To achieve this, firstly, we dynamically sample random positions within the images as assumed keypoints during training, allowing the network to explore a broader matching space. Secondly, we replace specific descriptors with high-efficiency sparse embeddings at multi levels of the image, facilitating the direct learning of underlying textures. Thirdly, we propose a novel and promising architecture, called Proposal-Guided TRansformer (PGTR), which aggregates context information from neighboring match proposals instead of searching globally with local features. PGTR works especially well under our training approach, and attain a synergistic advantage in terms of performance and efficiency. The overall pipeline achieves outstanding performance on various keypoints without any retraining, and can be flexibly reused when new keypoints emerge, making it valuable for real-world applications. Code will be available.Keywords
Funding Information
- National Natural Science Foundation of China (62171281)
- Science and Technology Commission of Shanghai Municipality (20DZ1200203)
- Science and Technology Commission of Shanghai Municipality (2021SHZDZX0102)
- Science and Technology Commission of Shanghai Municipality (22DZ2229005)
This publication has 54 references indexed in Scilit:
- MatchFormer: Interleaving Attention in Transformers for Feature Matching2023
- ASpanFormer: Detector-Free Image Matching with Adaptive Span Transformer2022
- Learning Feature Descriptors Using Camera Pose Supervision2020
- Image Matching from Handcrafted to Deep Features: A SurveyInternational Journal of Computer Vision, 2020
- Object Pose Estimation from Monocular Image Using Multi-view Keypoint Correspondence2019
- KAZE Features2012
- BRIEF: Binary Robust Independent Elementary Features2010
- Machine Learning for High-Speed Corner Detection2006
- SURF: Speeded Up Robust Features2006
- Distinctive Image Features from Scale-Invariant KeypointsInternational Journal of Computer Vision, 2004