OmniGlue: Generalizable Feature Matching with Foundation Model Guidance
Preprint
Abstract
The image matching field has been witnessing a continuous emergence of novel learnable feature matching techniques, with ever-improving performance on conventional benchmarks. However, our investigation shows that despite these gains, their potential for real-world applications is restricted by their limited generalization capabilities to novel image domains. In this paper, we introduce OmniGlue, the first learnable image matcher that is designed with generalization as a core principle. OmniGlue leverages broad knowledge from a vision foundation model to guide the feature matching process, boosting generalization to domains not seen at training time. Additionally, we propose a novel keypoint position-guided attention mechanism which disentangles spatial and appearance information, leading to enhanced matching descriptors. We perform comprehensive experiments on a suite of $7$ datasets with varied image domains, including scene-level, object-centric and aerial images. OmniGlue's novel components lead to relative gains on unseen domains of $20.9\%$ with respect to a directly comparable reference model, while also outperforming the recent LightGlue method by $9.5\%$ relatively.Code and model can be found at https://hwjiang1510.github.io/OmniGlue
Keywords
All Related Versions
This publication has 11 references indexed in Scilit:
- Doduo: Learning Dense Visual Correspondence from Unsupervised Semantic-Aware Flow2024
- DeDoDe: Detect, Don’t Describe — Describe, Don’t Detect for Local Feature Matching2024
- LFM-3D: Learnable Feature Matching Across Wide Baselines Using 3D Signals2024
- Few-View Object Reconstruction with Unknown Categories and Camera Poses2024
- PDC-Net+: Enhanced Probabilistic Dense Correspondence NetworkIEEE Transactions on Pattern Analysis and Machine Intelligence, 2023
- RayTran: 3D Pose Estimation and Shape Reconstruction of Multiple Objects from Videos with Ray-Traced Transformers2022
- Image Matching from Handcrafted to Deep Features: A SurveyInternational Journal of Computer Vision, 2020
- Past, Present, and Future of Simultaneous Localization and Mapping: Toward the Robust-Perception AgeIEEE Transactions on Robotics, 2016
- Random Sample Consensus: A Paradigm for Model Fitting with Applications to Image Analysis and Automated Cartography1987
- Concerning nonnegative matrices and doubly stochastic matricesPacific Journal of Mathematics, 1967