Structure-Preserving Frequency-Regularized Text-Guided Optimal Transport for Unpaired Rain Streaks and Raindrops Removal
Top Cited Papers
- 12 January 2026
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Multimedia
- Vol. 28 (15209210) , 3380-3395
Abstract
The removal of rain streaks and raindrops is crucial for enhancing the image visibility and mitigating the weather degradations. However, most existing approaches rely on the paired rainy and clean images, which are challenging to obtain in real-world scenarios. To this end, we propose a novel structure-preserving frequency-regularized text-guided optimal transport (SFTOT) framework, which formulates the unpaired rain streaks and raindrops removal as an optimal transport problem. Specifically, we introduce a structure-preserving transport cost, incorporating the structural similarity constraint to minimize the duality gap between the primal and dual formulations, while preserving the structural details of reconstructed images. Furthermore, by embedding the inherent frequency sparsity of rain streaks and raindrops into the transport cost, we derive a frequency-regularized optimal transport objective, ensuring consistency in frequency distributions between the generated and clean images. Additionally, we employ a pre-trained one-step stable diffusion model as the restoration network, which is fine-tuned using the low-rank adaptation (LoRA) adapters and zero convolutional layers, while integrating the domain-specific text prompts for both degraded and clean images to guide the generation process. Extensive experiments demonstrate that our method surpasses the existing well-performing unpaired learning approaches, achieving notable improvements in both the fidelity and photo-realism.Keywords
Funding Information
- National Natural Science Foundation of China (52441205)
- National Natural Science Foundation of China (52572353)
- National R&D Program of China (2023YFB2504703)
- Shaanxi International S&T Cooperation Program Project (2024GH-YBXM-24)
- Key R&D Program of Shaanxi Province (2024GX-YBXM-015)
This publication has 81 references indexed in Scilit:
- Adversarial Diffusion Distillation2024
- Neural Schrödinger bridge for unpaired real-world image derainingInformation Sciences, 2024
- PhDnet: A novel physic-aware dehazing network for remote sensing imagesInformation Fusion, 2024
- From heavy rain removal to detail restoration: A faster and better networkPattern Recognition, 2023
- Robust Unpaired Image Dehazing via Density and Depth DecompositionInternational Journal of Computer Vision, 2023
- OTRE: Where Optimal Transport Guided Unpaired Image-to-Image Translation Meets Regularization by Enhancing2023
- UnfairGAN: An enhanced generative adversarial network for raindrop removal from a single imageExpert Systems with Applications, 2022
- You Only Look Yourself: Unsupervised and Untrained Single Image Dehazing Neural NetworkInternational Journal of Computer Vision, 2021
- Contrastive Learning for Unpaired Image-to-Image Translation2020
- Optimal Transport2009