General Vision Encoder Features as Guidance in Medical Image Registration
Preprint
Abstract
General vision encoders like DINOv2 and SAM have recently transformed computer vision. Even though they are trained on natural images, such encoder models have excelled in medical imaging, e.g., in classification, segmentation, and registration. However, no in-depth comparison of different state-of-the-art general vision encoders for medical registration is available. In this work, we investigate how well general vision encoder features can be used in the dissimilarity metrics for medical image registration. We explore two encoders that were trained on natural images as well as one that was fine-tuned on medical data. We apply the features within the well-established B-spline FFD registration framework. In extensive experiments on cardiac cine MRI data, we find that using features as additional guidance for conventional metrics improves the registration quality. The code is available at github.com/compai-lab/2024-miccai-koegl.Keywords
All Related Versions
This publication has 21 references indexed in Scilit:
- Efficient multimodal large language models: a surveyVisual Intelligence, 2025
- Leveraging foundation models for content-based image retrieval in radiologyComputers in Biology and Medicine, 2025
- uniGradICON: A Foundation Model for Medical Image Registration2024
- Can surgical computer vision benefit from large-scale visual foundation models?International Journal of Computer Assisted Radiology and Surgery, 2024
- Surgical-DINO: adapter learning of foundation models for depth estimation in endoscopic surgeryInternational Journal of Computer Assisted Radiology and Surgery, 2024
- Deep learning in medical image registration: a surveyMachine Vision and Applications, 2020
- Learning deep similarity metric for 3D MR–TRUS image registrationInternational Journal of Computer Assisted Radiology and Surgery, 2018
- A Deep Metric for Multimodal Registration2016
- MIND: Modality independent neighbourhood descriptor for multi-modal deformable registrationMedical Image Analysis, 2012
- Symmetric diffeomorphic image registration with cross-correlation: Evaluating automated labeling of elderly and neurodegenerative brainMedical Image Analysis, 2008