Joint Conditional Diffusion Model for Image Restoration with Mixed Degradations
Authors:
- Yufeng YueYYYufeng Yue
- Meng YuMYMeng Yu
- Luojie YangLYLuojie Yang
- Yi YangYYYi Yang
Preprint
Abstract
Image restoration is rather challenging in adverse weather conditions, especially when multiple degradations occur simultaneously. Blind image decomposition was proposed to tackle this issue, however, its effectiveness heavily relies on the accurate estimation of each component. Although diffusion-based models exhibit strong generative abilities in image restoration tasks, they may generate irrelevant contents when the degraded images are severely corrupted. To address these issues, we leverage physical constraints to guide the whole restoration process, where a mixed degradation model based on atmosphere scattering model is constructed. Then we formulate our Joint Conditional Diffusion Model (JCDM) by incorporating the degraded image and degradation mask to provide precise guidance. To achieve better color and detail recovery results, we further integrate a refinement network to reconstruct the restored image, where Uncertainty Estimation Block (UEB) is employed to enhance the features. Extensive experiments performed on both multi-weather and weather-specific datasets demonstrate the superiority of our method over state-of-the-art competing methods.Keywords
All Related Versions
This publication has 10 references indexed in Scilit:
- Deep Adaptive Pansharpening via Uncertainty-Aware Image FusionIEEE Transactions on Geoscience and Remote Sensing, 2023
- Diffusion Models in Vision: A SurveyIEEE Transactions on Pattern Analysis and Machine Intelligence, 2023
- Vision Transformers for Single Image DehazingIEEE Transactions on Image Processing, 2023
- Restoring Vision in Adverse Weather Conditions With Patch-Based Denoising Diffusion ModelsIEEE Transactions on Pattern Analysis and Machine Intelligence, 2023
- RCDNet: An Interpretable Rain Convolutional Dictionary Network for Single Image DerainingIEEE Transactions on Neural Networks and Learning Systems, 2023
- Deep Dense Multi-Scale Network for Snow Removal Using Semantic and Depth PriorsIEEE Transactions on Image Processing, 2021
- Uncertainty Guided Multi-Scale Attention Network for Raindrop Removal From a Single ImageIEEE Transactions on Image Processing, 2021
- DesnowNet: Context-Aware Deep Network for Snow RemovalIEEE Transactions on Image Processing, 2018
- A Generalization of Algebraic Surface DrawingACM Transactions on Graphics, 1982
- Optics of the Atmosphere: Scattering by Molecules and ParticlesPhysics Today, 1977