RainDiffusion:When Unsupervised Learning Meets Diffusion Models for Real-world Image Deraining
Authors:
Preprint
Abstract
What will happen when unsupervised learning meets diffusion models for real-world image deraining? To answer it, we propose RainDiffusion, the first unsupervised image deraining paradigm based on diffusion models. Beyond the traditional unsupervised wisdom of image deraining, RainDiffusion introduces stable training of unpaired real-world data instead of weakly adversarial training. RainDiffusion consists of two cooperative branches: Non-diffusive Translation Branch (NTB) and Diffusive Translation Branch (DTB). NTB exploits a cycle-consistent architecture to bypass the difficulty in unpaired training of standard diffusion models by generating initial clean/rainy image pairs. DTB leverages two conditional diffusion modules to progressively refine the desired output with initial image pairs and diffusive generative prior, to obtain a better generalization ability of deraining and rain generation. Rain-Diffusion is a non adversarial training paradigm, serving as a new standard bar for real-world image deraining. Extensive experiments confirm the superiority of our RainDiffusion over un/semi-supervised methods and show its competitive advantages over fully-supervised ones.Keywords
All Related Versions
This publication has 9 references indexed in Scilit:
- Restoring Vision in Adverse Weather Conditions With Patch-Based Denoising Diffusion ModelsIEEE Transactions on Pattern Analysis and Machine Intelligence, 2023
- Image Super-Resolution Via Iterative RefinementIEEE Transactions on Pattern Analysis and Machine Intelligence, 2022
- DerainCycleGAN: Rain Attentive CycleGAN for Single Image Deraining and RainmakingIEEE Transactions on Image Processing, 2021
- Deep Learning for Generic Object Detection: A SurveyInternational Journal of Computer Vision, 2019
- Unsupervised Single Image Deraining with Self-Supervised Constraints2019
- The Devil is in the Decoder: Classification, Regression and GANsInternational Journal of Computer Vision, 2019
- A Hierarchical Approach for Rain or Snow Removing in a Single Color ImageIEEE Transactions on Image Processing, 2017
- Clearing the Skies: A Deep Network Architecture for Single-Image Rain RemovalIEEE Transactions on Image Processing, 2017
- Automatic Single-Image-Based Rain Streaks Removal via Image DecompositionIEEE Transactions on Image Processing, 2011