Self-Reference Image Super-Resolution via Pre-trained Diffusion Large Model and Window Adjustable Transformer
Open Access
- 26 October 2023
- conference paper
- Published by Association for Computing Machinery (ACM)
- p. 7981-7992
Abstract
Currently, reference-based super-resolution (RefSR) techniques leverage high-resolution (HR) reference images to provide useful content and texture information for low-resolution (LR) images during the super-resolution (SR) process. Nevertheless, it is time-consuming, laborious, and even impossible in some cases to find high-quality reference images. To tackle this problem, we propose a brand-new self-reference image super-resolution approach using a pre-trained diffusion large model and a window adjustable transformer, termed DWTrans. Our proposed method does not require explicitly inputting manually acquired reference images during training and inference. Specifically, we feed the degraded LR images into a pre-trained stable diffusion large model to automatically generate corresponding high-quality self-reference (SRef) images that provide valuable high-frequency details for the LR images in the process of SR. To extract valuable high-frequency information in SRef images, we design a window adjustable transformer with both non-adjustable window layer (NWL) and adjustable window layer (AWL). The NWL learns local features from LR images using a dense window, while the AWL acquires global features from the SRef images using a random sparse window. Furthermore, to fully utilize the high-frequency features in the SRef image, we introduce the adaptive deformable fusion module to adaptively fuse the features of the LR and SRef images. Experimental results validate that our proposed DWTrans outperforms state-of-the-art methods on various benchmark datasets both quantitatively and visually.Keywords
Funding Information
- Zhejiang Elite Program (2022C01222)
- National Program of China (2020YFC1523201, 62172365,19ZDA197)
- Key Technologies and Product Research and Development Projects for Cultural Relics Protection and Trading Circulation
This publication has 38 references indexed in Scilit:
- Image Super-Resolution Using Very Deep Residual Channel Attention Networks2018
- CrossNet: An End-to-End Reference-Based Super Resolution Network Using Cross-Scale Warping2018
- NTIRE 2017 Challenge on Single Image Super-Resolution: Methods and Results2017
- Enhanced Deep Residual Networks for Single Image Super-Resolution2017
- Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network2017
- Sketch-based manga retrieval using manga109 datasetMultimedia Tools and Applications, 2016
- Single image super-resolution from transformed self-exemplars2015
- Low-Complexity Single-Image Super-Resolution based on Nonnegative Neighbor Embedding2012
- Pointwise Shape-Adaptive DCT for High-Quality Denoising and Deblocking of Grayscale and Color ImagesIEEE Transactions on Image Processing, 2007
- A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics2002