A least square generative network based on invariant contrastive feature pair learning for multimodal MR image synthesis
- 27 April 2023
- journal article
- research article
- Published by Springer Nature in International Journal of Computer Assisted Radiology and Surgery
- Vol. 18 (6) , 971-979
Abstract
Purpose During MR-guided neurosurgical procedures, several factors may limit the acquisition of additional MR sequences, which are needed by neurosurgeons to adjust surgical plans or ensure complete tumor resection. Automatically synthesized MR contrasts generated from other available heterogeneous MR sequences could alleviate timing constraints. Methods We propose a new multimodal MR synthesis approach leveraging a combination of MR modalities presenting glioblastomas to generate an additional modality. The proposed learning approach relies on a least square GAN (LSGAN) using an unsupervised contrastive learning strategy. We incorporate a contrastive encoder, which extracts an invariant contrastive representation from augmented pairs of the generated and real target MR contrasts. This contrastive representation describes a pair of features for each input channel, allowing to regularize the generator to be invariant to the high-frequency orientations. Moreover, when training the generator, we impose on the LSGAN loss another term reformulated as the combination of a reconstruction and a novel perception loss based on a pair of features. Results When compared to other multimodal MR synthesis approaches evaluated on the BraTS’18 brain dataset, the model yields the highest Dice score with \(0.748\,\pm \,0.04\) and achieves the lowest variability information of \(2.1\,\pm \,1.11\), with a probability rand index score of \(0.84\,\pm \,0.03\) and a global consistency error of \(0.17\,\pm \,0.04\). Conclusion The proposed model allows to generate reliable MR contrasts with enhanced tumors on the synthesized image using a brain tumor dataset (BraTS’18). In future work, we will perform a clinical evaluation of residual tumor segmentations during MR-guided neurosurgeries, where limited MR contrasts will be acquired during the procedure.
Keywords
Funding Information
- Natural Sciences and Engineering Research Council of Canada
This publication has 12 references indexed in Scilit:
- Five-Minute Five-Sequence Knee MRI Using Combined Simultaneous Multislice and Parallel Imaging Acceleration: Comparison with 10-Minute Parallel Imaging Knee MRIRadiology, 2021
- A Unified Conditional Disentanglement Framework For Multimodal Brain Mr Image TranslationIEEE International Symposium on Biomedical Imaging, 2021
- Multi-Modality MR Image Synthesis via Confidence-Guided Aggregation and Cross-Modality RefinementIEEE Journal of Biomedical and Health Informatics, 2021
- A feature invariant generative adversarial network for head and neck MRI/CT image synthesisPhysics in Medicine & Biology, 2021
- Hi-Net: Hybrid-Fusion Network for Multi-Modal MR Image SynthesisIEEE Transactions on Medical Imaging, 2020
- Generative Adversarial Training for MRA Image Synthesis Using Multi-contrast MRI2018
- Multimodal MR Synthesis via Modality-Invariant Latent RepresentationIEEE Transactions on Medical Imaging, 2017
- Least Squares Generative Adversarial Networks2017
- Intraoperative MR‐guided neurosurgeryJournal of Magnetic Resonance Imaging, 2008
- DEEP BRAIN STIMULATIONAnnual Review of Neuroscience, 2006