Rigid Single-Slice-in-Volume registration via rotation-equivariant 2D/3D feature matching
Authors:
- Stefan BrandstätterSBStefan Brandstätter
- Philipp SeeböckPSPhilipp Seeböck
- Christoph FürböckCFChristoph Fürböck
- Svitlana PochepniaSPSvitlana Pochepnia
- Helmut ProschHPHelmut Prosch
- Georg LangsGLGeorg Langs
Preprint
Abstract
2D to 3D registration is essential in tasks such as diagnosis, surgical navigation, environmental understanding, navigation in robotics, autonomous systems, or augmented reality. In medical imaging, the aim is often to place a 2D image in a 3D volumetric observation to w. Current approaches for rigid single slice in volume registration are limited by requirements such as pose initialization, stacks of adjacent slices, or reliable anatomical landmarks. Here, we propose a self-supervised 2D/3D registration approach to match a single 2D slice to the corresponding 3D volume. The method works in data without anatomical priors such as images of tumors. It addresses the dimensionality disparity and establishes correspondences between 2D in-plane and 3D out-of-plane rotation-equivariant features by using group equivariant CNNs. These rotation-equivariant features are extracted from the 2D query slice and aligned with their 3D counterparts. Results demonstrate the robustness of the proposed slice-in-volume registration on the NSCLC-Radiomics CT and KIRBY21 MRI datasets, attaining an absolute median angle error of less than 2 degrees and a mean-matching feature accuracy of 89% at a tolerance of 3 pixels.Keywords
All Related Versions
- Version 1, 2024-10-24, ArXiv
- Published version: , 280.
This publication has 17 references indexed in Scilit:
- A robust and interpretable deep learning framework for multi-modal registration via keypointsMedical Image Analysis, 2023
- Deep learning-based 2D/3D registration of an atlas to biplanar X-ray imagesInternational Journal of Computer Assisted Radiology and Surgery, 2022
- Robust joint registration of multiple stains and MRI for multimodal 3D histology reconstruction: Application to the Allen human brain atlasMedical Image Analysis, 2021
- A fast object registration method for augmented reality assembly with simultaneous determination of multiple 2D-3D correspondencesRobotics and Computer-Integrated Manufacturing, 2020
- Using DeepLabCut for 3D markerless pose estimation across species and behaviorsNature Protocols, 2019
- Robust 3D–2D image registration: application to spine interventions and vertebral labeling in the presence of anatomical deformationPhysics in Medicine & Biology, 2013
- A review of 3D/2D registration methods for image-guided interventionsMedical Image Analysis, 2012
- Multi-parametric neuroimaging reproducibility: A 3-T resource studyNeuroImage, 2011
- Distinctive Image Features from Scale-Invariant KeypointsInternational Journal of Computer Vision, 2004
- Random Sample Consensus: A Paradigm for Model Fitting with Applications to Image Analysis and Automated Cartography1987