Let UNet Play an Adversarial Game: Investigating the Effect of Adversarial Training in Enhancing Low-Resolution MRI
- 31 July 2024
- journal article
- research article
- Published by Springer Nature in Journal of Imaging Informatics in Medicine
- Vol. 38 (1) , 629-645
Abstract
Adversarial training has attracted much attention in enhancing the visual realism of images, but its efficacy in clinical imaging has not yet been explored. This work investigated adversarial training in a clinical context, by training 206 networks on the OASIS-1 dataset for improving low-resolution and low signal-to-noise ratio (SNR) magnetic resonance images. Each network corresponded to a different combination of perceptual and adversarial loss weights and distinct learning rate values. For each perceptual loss weighting, we identified its corresponding adversarial loss weighting that minimized structural disparity. Each optimally weighted adversarial loss yielded an average SSIM reduction of 1.5%. We further introduced a set of new metrics to assess other clinically relevant image features: Gradient Error (GE) to measure structural disparities; Sharpness to compute edge clarity; and Edge-Contrast Error (ECE) to quantify any distortion of the pixel distribution around edges. Including adversarial loss increased structural enhancement in visual inspection, which correlated with statistically consistent GE reductions (p-value << 0.05). This also resulted in increased Sharpness; however, the level of statistical significance was dependent on the perceptual loss weighting. Additionally, adversarial loss yielded ECE reductions for smaller perceptual loss weightings, while showing non-significant increases (p-value >> 0.05) when these weightings were higher, demonstrating that the increased Sharpness does not adversely distort the pixel distribution around the edges in the image. These studies clearly suggest that adversarial training significantly improves the performance of an MRI enhancement pipeline, and highlights the need for systematic studies of hyperparameter optimization and investigation of alternative image quality metrics.Keywords
This publication has 77 references indexed in Scilit:
- Deep Learning Whole‐Gland and Zonal Prostate Segmentation on a Public MRI DatasetJournal of Magnetic Resonance Imaging, 2021
- Training a neural network for Gibbs and noise removal in diffusion MRIMagnetic Resonance in Medicine, 2020
- MRI reconstruction using deep Bayesian estimationMagnetic Resonance in Medicine, 2020
- Reconstruction of multicontrast MR images through deep learningMedical Physics, 2019
- Suppressing motion artefacts in MRI using an Inception‐ResNet network with motion simulation augmentationNMR in Biomedicine, 2019
- X‐nuclei imaging: Current state, technical challenges, and future directionsJournal of Magnetic Resonance Imaging, 2019
- ESRGAN: Enhanced Super-Resolution Generative Adversarial Networks2019
- Efficient and Accurate MRI Super-Resolution Using a Generative Adversarial Network and 3D Multi-level Densely Connected Network2018
- Edge sharpness assessment by parametric modeling: Application to magnetic resonance imagingConcepts in Magnetic Resonance Part A, 2015
- Undersampled projection reconstruction applied to MR angiographyMagnetic Resonance in Medicine, 2000