RainDiffusion: When Unsupervised Learning Meets Diffusion Models for Real-world Image Deraining
Authors:
Preprint
Abstract
Recent diffusion models show great potential in generative modeling tasks. This motivates us to raise an intriguing question - What will happen when unsupervised learning meets diffusion models for real-world image deraining? Before answering it, we observe two major obstacles of diffusion models in real-world image deraining: the need for paired training data and the limited utilization of multi-scale rain patterns. To overcome the obstacles, we propose RainDiffusion, the first real-world image deraining paradigm based on diffusion models. RainDiffusion is a non-adversarial training paradigm that introduces stable training of unpaired real-world data, rather than weakly adversarial training, serving as a new standard bar for real-world image deraining. It consists of two cooperative branches: Non-diffusive Translation Branch (NTB) and Diffusive Translation Branch (DTB). NTB exploits a cycle-consistent architecture to bypass the difficulty in unpaired training of regular diffusion models by generating initial clean/rainy image pairs. Given initial image pairs, DTB leverages multi-scale diffusion models to progressively refine the desired output via diffusive generative and multi-scale priors, to obtain a better generalization capacity of real-world image deraining. Extensive experiments confirm the superiority of our RainDiffusion over eight un/semi-supervised methods and show its competitive advantages over seven fully-supervised ones.Keywords
All Related Versions
This publication has 9 references indexed in Scilit:
- Restoring Vision in Adverse Weather Conditions With Patch-Based Denoising Diffusion ModelsIEEE Transactions on Pattern Analysis and Machine Intelligence, 2023
- Image Super-Resolution Via Iterative RefinementIEEE Transactions on Pattern Analysis and Machine Intelligence, 2022
- DerainCycleGAN: Rain Attentive CycleGAN for Single Image Deraining and RainmakingIEEE Transactions on Image Processing, 2021
- Deep Learning for Generic Object Detection: A SurveyInternational Journal of Computer Vision, 2019
- Unsupervised Single Image Deraining with Self-Supervised Constraints2019
- The Devil is in the Decoder: Classification, Regression and GANsInternational Journal of Computer Vision, 2019
- A Hierarchical Approach for Rain or Snow Removing in a Single Color ImageIEEE Transactions on Image Processing, 2017
- Clearing the Skies: A Deep Network Architecture for Single-Image Rain RemovalIEEE Transactions on Image Processing, 2017
- Automatic Single-Image-Based Rain Streaks Removal via Image DecompositionIEEE Transactions on Image Processing, 2011