BSRT: Improving Burst Super-Resolution with Swin Transformer and Flow-Guided Deformable Alignment
Authors:
- 1 June 2022
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 997-1007
Abstract
This work addresses the Burst Super-Resolution (BurstSR) task using a new architecture, which requires restoring a high-quality image from a sequence of noisy, misaligned, and low-resolution RAW bursts. To over-come the challenges in BurstSR, we propose a Burst Super-Resolution Transformer (BSRT), which can significantly improve the capability of extracting inter-frame information and reconstruction. To achieve this goal, we propose a Pyramid Flow-Guided Deformable Convolution Network (Pyramid FG-DCN) and incorporate Swin Trans-former Blocks and Groups as our main backbone. More specifically, we combine optical flows and deformable convolutions, hence our BSRT can handle misalignment and aggregate the potential texture information in multi-frames more efficiently. In addition, our Transformer-based structure can capture long-range dependency to further improve the performance. The evaluation on both synthetic and real-world tracks demonstrates that our approach achieves a new state-of-the-art in BurstSR task. Further, our BSRT wins the championship in the NTIRE2022 Burst Super-Resolution Challenge.Keywords
All Related Versions
Funding Information
- National Natural Science Foundation of China
This publication has 41 references indexed in Scilit:
- Deeply-Recursive Convolutional Network for Image Super-Resolution2016
- Accurate Image Super-Resolution Using Very Deep Convolutional Networks2016
- Multi chaotic systems based pixel shuffle for image encryptionOptics Communications, 2009
- Kernel Regression for Image Processing and ReconstructionIEEE Transactions on Image Processing, 2007
- Image quality assessment: from error visibility to structural similarityIEEE Transactions on Image Processing, 2004
- High-resolution image reconstruction from a sequence of rotated and translated frames and its application to an infrared imaging systemOptical Engineering, 1998
- Restoration of a single superresolution image from several blurred, noisy, and undersampled measured imagesIEEE Transactions on Image Processing, 1997
- Motion deblurring and super-resolution from an image sequence1996
- Improving resolution by image registrationCVGIP: Graphical Models and Image Processing, 1991
- Improving image resolution using subpixel motionPattern Recognition Letters, 1987