TCSR: Lightweight Transformer and CNN Interaction Network for Image Super-Resolution
Open Access
- 8 October 2024
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 12, 174782-174795
Abstract
Convolutional neural network (CNN) has achieved impressive success in lightweight image super-resolution (SR) methods, yet the nature of its local operations constrains the SR performance. Recent Transformer has attracted increasing attention in lightweight SR methods owing to its remarkable global feature extraction capacity. However, the huge computational cost makes it challenging for lightweight SR methods to efficiently utilize Transformer to exploit global contextual information from shallowto intermediate layers. To address these issues, we propose a novel lightweight Transformer and CNN interaction network for image Super-Resolution (TCSR), which fully leverages the complementary strengths of Transformer and CNN. Specifically, an efficient lightweight Transformer and CNN Interaction Block (TCIB) is designed to extract local and global features at various stages of the network, resulting in favorable hybrid features that significantly improve the quality of reconstructed images. Then, we construct a lightweight Reversed UNet (RUNet) to progressively aggregate hybrid features as well as to better trade-off the reconstruction accuracy and efficiency. Furthermore, we introduce a Refinement module to further refine edge and texture details with global information. Experimental results on numerous benchmarks validate that the proposed TCSR achieves superior performance with fewer parameters and less computational overhead than state-of-the-art lightweight methods.Keywords
Funding Information
- National Natural Science Foundation of China (61901183)
- Natural Science Foundation of Fujian Province (2023J01140)
- Natural Science Foundation of Fujian Provincial Science and Technology Department (2021H6037, 2023H6034)
- Key Project of Quanzhou Science and Technology Plan (2021C008R, 2023C007R)
- 2022 National College Student Innovation and Entrepreneurship Training Program Project (202210399010)
- Sixth Batch of Quanzhou City’s Introduction of High-Level Talent Team Projects No2
This publication has 49 references indexed in Scilit:
- Unsupervised multi-level spatio-spectral fusion transformer for hyperspectral image super-resolutionOptics & Laser Technology, 2024
- Lightweight refined networks for single image super-resolutionMultimedia Tools and Applications, 2021
- FSFN: feature separation and fusion network for single image super-resolutionMultimedia Tools and Applications, 2021
- End-to-End Object Detection with Transformers2020
- Fast, Accurate, and Lightweight Super-Resolution with Cascading Residual Network2018
- Image Super-Resolution Using Very Deep Residual Channel Attention Networks2018
- Sketch-based manga retrieval using manga109 datasetMultimedia Tools and Applications, 2016
- Accelerating the Super-Resolution Convolutional Neural Network2016
- U-Net: Convolutional Networks for Biomedical Image Segmentation2015
- Single image super-resolution from transformed self-exemplars2015