Single image super-resolution with denoising diffusion GANS
Open Access
- 21 February 2024
- journal article
- research article
- Published by Springer Nature in Scientific Reports
- Vol. 14 (1) , 1-18
Abstract
Single image super-resolution (SISR) refers to the reconstruction from the corresponding low-resolution (LR) image input to a high-resolution (HR) image. However, since a single low-resolution image corresponds to multiple high-resolution images, this is an ill-posed problem. In recent years, generative model-based SISR methods have outperformed conventional SISR methods in performance. However, the SISR methods based on GAN, VAE, and Flow have the problems of unstable training, low sampling quality, and expensive computational cost. These models also struggle to achieve the trifecta of diverse, high-quality, and fast sampling. In particular, denoising diffusion probabilistic models have shown impressive variety and high quality of samples, but their expensive sampling cost prevents them from being well applied in the real world. In this paper, we investigate the fundamental reason for the slow sampling speed of the SISR method based on the diffusion model lies in the Gaussian assumption used in the previous diffusion model, which is only applicable for small step sizes. We propose a new Single Image Super-Resolution with Denoising Diffusion GANS (SRDDGAN) to achieve large-step denoising, sample diversity, and training stability. Our approach combines denoising diffusion models with GANs to generate images conditionally, using a multimodal conditional GAN to model each denoising step. SRDDGAN outperforms existing diffusion model-based methods regarding PSNR and perceptual quality metrics, while the added latent variable Z solution explores the diversity of likely HR spatial domain. Notably, the SRDDGAN model infers nearly 11 times faster than diffusion-based SR3, making it a more practical solution for real-world applications.Keywords
Funding Information
- Science and Technology Major Project of Guangxi (AA19254016, AA19254016, AA19254016, AA19254016, AA19254016)
- Beihai city science and technology planning project (202082033, 202082033, 202082033, 202082033, 202082033)
- Guangxi Graduate Student Innovation Project (YCSW2021174, YCSW2021174, YCSW2021174, YCSW2021174, YCSW2021174)
This publication has 50 references indexed in Scilit:
- Medical image segmentation model based on triple gate MultiLayer perceptronScientific Reports, 2022
- SRDiff: Single image super-resolution with diffusion probabilistic modelsNeurocomputing, 2022
- Task-Driven Super Resolution: Object Detection in Low-Resolution Images2021
- SRFlow: Learning the Super-Resolution Space with Normalizing Flow2020
- Feedback Network for Image Super-Resolution2019
- The Unreasonable Effectiveness of Deep Features as a Perceptual Metric2018
- Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network2017
- Simultaneous Super-Resolution and Cross-Modality Synthesis of 3D Medical Images Using Weakly-Supervised Joint Convolutional Sparse Coding2017
- Single-frame super-resolution in remote sensing: a practical overviewInternational Journal of Remote Sensing, 2016
- Convolutional Neural Network Super Resolution for Face Recognition in Surveillance Monitoring2016