Teaching Tailored to Talent: Adverse Weather Restoration via Prompt Pool and Depth-Anything Constraint
Authors:
Preprint
Abstract
Recent advancements in adverse weather restoration have shown potential, yet the unpredictable and varied combinations of weather degradations in the real world pose significant challenges. Previous methods typically struggle with dynamically handling intricate degradation combinations and carrying on background reconstruction precisely, leading to performance and generalization limitations. Drawing inspiration from prompt learning and the "Teaching Tailored to Talent" concept, we introduce a novel pipeline, T3-DiffWeather. Specifically, we employ a prompt pool that allows the network to autonomously combine sub-prompts to construct weather-prompts, harnessing the necessary attributes to adaptively tackle unforeseen weather input. Moreover, from a scene modeling perspective, we incorporate general prompts constrained by Depth-Anything feature to provide the scene-specific condition for the diffusion process. Furthermore, by incorporating contrastive prompt loss, we ensures distinctive representations for both types of prompts by a mutual pushing strategy. Experimental results demonstrate that our method achieves state-of-the-art performance across various synthetic and real-world datasets, markedly outperforming existing diffusion techniques in terms of computational efficiency.Keywords
All Related Versions
This publication has 25 references indexed in Scilit:
- DiffBIR: Toward Blind Image Restoration with Generative Diffusion Prior2024
- Pixel-Aware Stable Diffusion for Realistic Image Super-Resolution and Personalized Stylization2024
- Spiking Wavelet Transformer2024
- See and Think: Embodied Agent in Virtual Environment2024
- Genuine Knowledge from Practice: Diffusion Test-Time Adaptation for Video Adverse Weather Removal2024
- Depth Anything: Unleashing the Power of Large-Scale Unlabeled Data2024
- Dual-former: Hybrid self-attention transformer for efficient image restorationDigital Signal Processing, 2024
- Snow Mask Guided Adaptive Residual Network for Image Snow RemovalComputer Vision and Image Understanding, 2023
- Learning to Prompt for Vision-Language ModelsInternational Journal of Computer Vision, 2022
- SRDiff: Single image super-resolution with diffusion probabilistic modelsNeurocomputing, 2022