Synth-by-Reg (SbR): Contrastive Learning for Synthesis-Based Registration of Paired Images
- 21 September 2021
- book chapter
- Published by Springer Nature
- Vol. 12965, 44-54
Abstract
Nonlinear inter-modality registration is often challenging due to the lack of objective functions that are good proxies for alignment. Here we propose a synthesis-by-registration method to convert this problem into an easier intra-modality task. We introduce a registration loss for weakly supervised image translation between domains that does not require perfectly aligned training data. This loss capitalises on a registration U-Net with frozen weights, to drive a synthesis CNN towards the desired translation. We complement this loss with a structure preserving constraint based on contrastive learning, which prevents blurring and content shifts due to overfitting. We apply this method to the registration of histological sections to MRI slices, a key step in 3D histology reconstruction. Results on two public datasets show improvements over registration based on mutual information (13% reduction in landmark error) and synthesis-based algorithms such as CycleGAN (11% reduction), and are comparable to registration with label supervision. Code and data are publicly available at https://github.com/acasamitjana/SynthByReg .Keywords
All Related Versions
This publication has 35 references indexed in Scilit:
- BigBrain: An Ultrahigh-Resolution 3D Human Brain ModelScience, 2013
- MIND: Modality independent neighbourhood descriptor for multi-modal deformable registrationMedical Image Analysis, 2012
- Within-subject template estimation for unbiased longitudinal image analysisNeuroImage, 2012
- MR to ultrasound registration for image-guided prostate interventionsMedical Image Analysis, 2012
- Unbiased average age-appropriate atlases for pediatric studiesNeuroImage, 2011
- Fast free-form deformation using graphics processing unitsComputer Methods and Programs in Biomedicine, 2010
- Unified segmentationNeuroImage, 2005
- Evaluation of atlas selection strategies for atlas-based image segmentation with application to confocal microscopy images of bee brainsNeuroImage, 2004
- Medical image registration using mutual informationProceedings of the IEEE, 2003
- Voxel-Based Morphometry—The MethodsNeuroImage, 2000