UCIP: A Universal Framework for Compressed Image Super-Resolution using Dynamic Prompt
Preprint
Abstract
Compressed Image Super-resolution (CSR) aims to simultaneously super-resolve the compressed images and tackle the challenging hybrid distortions caused by compression. However, existing works on CSR usually focuses on a single compression codec, i.e., JPEG, ignoring the diverse traditional or learning-based codecs in the practical application, e.g., HEVC, VVC, HIFIC, etc. In this work, we propose the first universal CSR framework, dubbed UCIP, with dynamic prompt learning, intending to jointly support the CSR distortions of any compression codecs/modes. Particularly, an efficient dynamic prompt strategy is proposed to mine the content/spatial-aware task-adaptive contextual information for the universal CSR task, using only a small amount of prompts with spatial size 1x1. To simplify contextual information mining, we introduce the novel MLP-like framework backbone for our UCIP by adapting the Active Token Mixer (ATM) to CSR tasks for the first time, where the global information modeling is only taken in horizontal and vertical directions with offset prediction. We also build an all-in-one benchmark dataset for the CSR task by collecting the datasets with the popular 6 diverse traditional and learning-based codecs, including JPEG, HEVC, VVC, HIFIC, etc., resulting in 23 common degradations. Extensive experiments have shown the consistent and excellent performance of our UCIP on universal CSR tasks. The project can be found in https://lixinustc.github.io/UCIP.github.ioKeywords
All Related Versions
This publication has 22 references indexed in Scilit:
- Diffusion Models for Image Restoration and Enhancement: A Comprehensive SurveyInternational Journal of Computer Vision, 2025
- Exploiting Diffusion Prior for Real-World Image Super-ResolutionInternational Journal of Computer Vision, 2024
- PromptCIR: Blind Compressed Image Restoration with Prompt Learning2024
- Multimodal Prompt Perceiver: Empower Adaptiveness, Generalizability and Fidelity for All-in-One Image Restoration2024
- CoSeR: Bridging Image and Language for Cognitive Super-Resolution2024
- SeeSR: Towards Semantics-Aware Real-World Image Super-Resolution2024
- CSwin2SR: Circular Swin2SR for Compressed Image Super-Resolution2023
- HST: Hierarchical Swin Transformer for Compressed Image Super-Resolution2023
- Swin2SR: SwinV2 Transformer for Compressed Image Super-Resolution and Restoration2023
- Sketch-based manga retrieval using manga109 datasetMultimedia Tools and Applications, 2016