RZSR: Reference-Based Zero-Shot Super-Resolution With Depth Guided Self-Exemplars
- 26 August 2022
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Multimedia
- Vol. 25 (15209210) , 5972-5983
Abstract
Recent methods for single image super-resolution (SISR) have demonstrated outstanding performance in generating high-resolution (HR) images from low-resolution (LR) images. However, most of these methods show their superiority using synthetically generated LR images, and their generalizability to real-world images is often not satisfactory. In this paper, we pay attention to two well-known strategies developed for robust super-resolution (SR), i.e., reference-based SR (RefSR) and zero-shot SR (ZSSR), and propose an integrated solution, called reference-based zero-shot SR (RZSR). Following the principle of ZSSR, we train an image-specific SR network at test time using training samples extracted only from the input image itself. To advance ZSSR, we obtain reference image patches with rich textures and high-frequency details which are also extracted only from the input image using cross-scale matching. To this end, we construct an internal reference dataset and retrieve reference image patches from the dataset using depth information. Using LR patches and their corresponding HR reference patches, we train a RefSR network that is embodied with a non-local attention module. Experimental results demonstrate the superiority of the proposed RZSR compared to the previous ZSSR methods and robustness to unseen images compared to other fully supervised SISR methods.Keywords
All Related Versions
Funding Information
- National Research Foundation of Korea (2020R1A4A4079705)
This publication has 39 references indexed in Scilit:
- Single Image Super-Resolution via Adaptive Transform-Based Nonlocal Self-Similarity Modeling and Learning-Based Gradient RegularizationIEEE Transactions on Multimedia, 2017
- Accurate Image Super-Resolution Using Very Deep Convolutional Networks2016
- Single image super-resolution from transformed self-exemplars2015
- Learning a Deep Convolutional Network for Image Super-Resolution2014
- SUN3D: A Database of Big Spaces Reconstructed Using SfM and Object Labels2013
- Landmark Image Super-Resolution by Retrieving Web ImagesIEEE Transactions on Image Processing, 2013
- A Category-Level 3D Object Dataset: Putting the Kinect to Work2013
- A Bayesian approach to adaptive video super resolution2011
- A simple and fast algorithm for K-medoids clusteringExpert Systems with Applications, 2009
- Image quality assessment: from error visibility to structural similarityIEEE Transactions on Image Processing, 2004