Rethinking Real-world Image Deraining via An Unpaired Degradation-Conditioned Diffusion Model
Authors:
Preprint
Abstract
Recent diffusion models have exhibited great potential in generative modeling tasks. Part of their success can be attributed to the ability of training stable on huge sets of paired synthetic data. However, adapting these models to real-world image deraining remains difficult for two aspects. First, collecting a large-scale paired real-world clean/rainy dataset is unavailable while regular conditional diffusion models heavily rely on paired data for training. Second, real-world rain usually reflects real-world scenarios with a variety of unknown rain degradation types, which poses a significant challenge for the generative modeling process. To meet these challenges, we propose RainDiff, the first real-world image deraining paradigm based on diffusion models, serving as a new standard bar for real-world image deraining. We address the first challenge by introducing a stable and non-adversarial unpaired cycle-consistent architecture that can be trained, end-to-end, with only unpaired data for supervision; and the second challenge by proposing a degradation-conditioned diffusion model that refines the desired output via a diffusive generative process conditioned by learned priors of multiple rain degradations. Extensive experiments confirm the superiority of our RainDiff over existing unpaired/semi-supervised methods and show its competitive advantages over several fully-supervised ones.Keywords
All Related Versions
This publication has 11 references indexed in Scilit:
- Image Super-Resolution Via Iterative RefinementIEEE Transactions on Pattern Analysis and Machine Intelligence, 2022
- DerainCycleGAN: Rain Attentive CycleGAN for Single Image Deraining and RainmakingIEEE Transactions on Image Processing, 2021
- Deep Learning for Generic Object Detection: A SurveyInternational Journal of Computer Vision, 2019
- Image De-Raining Using a Conditional Generative Adversarial NetworkIEEE Transactions on Circuits and Systems for Video Technology, 2019
- The Devil is in the Decoder: Classification, Regression and GANsInternational Journal of Computer Vision, 2019
- Benchmarking Single-Image Dehazing and BeyondIEEE Transactions on Image Processing, 2018
- A Hierarchical Approach for Rain or Snow Removing in a Single Color ImageIEEE Transactions on Image Processing, 2017
- Clearing the Skies: A Deep Network Architecture for Single-Image Rain RemovalIEEE Transactions on Image Processing, 2017
- Scope of validity of PSNR in image/video quality assessmentElectronics Letters, 2008
- Image quality assessment: from error visibility to structural similarityIEEE Transactions on Image Processing, 2004