Uni-COAL: A Unified Framework for Cross-Modality Synthesis and Super-Resolution of MR Images
Authors:
- Zhiyun SongZSZhiyun Song
- Zengxin QiZQZengxin Qi
- Xin WangXWXin Wang
- Xiangyu ZhaoXZXiangyu Zhao
- Zhenrong ShenZSZhenrong Shen
- Sheng WangSWSheng Wang
- Manman FeiMFManman Fei
- Zhe WangZWZhe Wang
- Di ZangDZDi Zang
- Dongdong ChenDCDongdong Chen
- Linlin YaoLYLinlin Yao
- Qian WangQWQian Wang
Preprint
Abstract
Cross-modality synthesis (CMS), super-resolution (SR), and their combination (CMSR) have been extensively studied for magnetic resonance imaging (MRI). Their primary goals are to enhance the imaging quality by synthesizing the desired modality and reducing the slice thickness. Despite the promising synthetic results, these techniques are often tailored to specific tasks, thereby limiting their adaptability to complex clinical scenarios. Therefore, it is crucial to build a unified network that can handle various image synthesis tasks with arbitrary requirements of modality and resolution settings, so that the resources for training and deploying the models can be greatly reduced. However, none of the previous works is capable of performing CMS, SR, and CMSR using a unified network. Moreover, these MRI reconstruction methods often treat alias frequencies improperly, resulting in suboptimal detail restoration. In this paper, we propose a Unified Co-Modulated Alias-free framework (Uni-COAL) to accomplish the aforementioned tasks with a single network. The co-modulation design of the image-conditioned and stochastic attribute representations ensures the consistency between CMS and SR, while simultaneously accommodating arbitrary combinations of input/output modalities and thickness. The generator of Uni-COAL is also designed to be alias-free based on the Shannon-Nyquist signal processing framework, ensuring effective suppression of alias frequencies. Additionally, we leverage the semantic prior of Segment Anything Model (SAM) to guide Uni-COAL, ensuring a more authentic preservation of anatomical structures during synthesis. Experiments on three datasets demonstrate that Uni-COAL outperforms the alternatives in CMS, SR, and CMSR tasks for MR images, which highlights its generalizability to wide-range applications.Keywords
All Related Versions
This publication has 28 references indexed in Scilit:
- Segment Anything Is Not Always Perfect: An Investigation of SAM on Different Real-world ApplicationsMachine Intelligence Research, 2024
- Arbitrary Reduction of MRI Inter-slice Spacing Using Hierarchical Feature Conditional Diffusion2023
- Understanding aliasing effects and their removal in SPEN MRI: A k‐space perspectiveMagnetic Resonance in Medicine, 2023
- AutoGAN-Synthesizer: Neural Architecture Search for Cross-Modality MRI Synthesis2022
- Progressively volumetrized deep generative models for data-efficient contextual learning of MR image recoveryMedical Image Analysis, 2022
- Three‐dimensional self‐attention conditional GAN with spectral normalization for multimodal neuroimaging synthesisMagnetic Resonance in Medicine, 2021
- Efficient and Accurate MRI Super-Resolution Using a Generative Adversarial Network and 3D Multi-level Densely Connected Network2018
- Super‐resolution musculoskeletal MRI using deep learningMagnetic Resonance in Medicine, 2018
- Deep Learning Based Imaging Data Completion for Improved Brain Disease Diagnosis2014
- A Compressed Sensing Approach for MR Tissue Contrast Synthesis2011