TCSR: Lightweight Transformer and CNN Interaction Network for Image Super-Resolution

Authors:

Abstract
Convolutional neural network (CNN) has achieved impressive success in lightweight image super-resolution (SR) methods, yet the nature of its local operations constrains the SR performance. Recent Transformer has attracted increasing attention in lightweight SR methods owing to its remarkable global feature extraction capacity. However, the huge computational cost makes it challenging for lightweight SR methods to efficiently utilize Transformer to exploit global contextual information from shallowto intermediate layers. To address these issues, we propose a novel lightweight Transformer and CNN interaction network for image Super-Resolution (TCSR), which fully leverages the complementary strengths of Transformer and CNN. Specifically, an efficient lightweight Transformer and CNN Interaction Block (TCIB) is designed to extract local and global features at various stages of the network, resulting in favorable hybrid features that significantly improve the quality of reconstructed images. Then, we construct a lightweight Reversed UNet (RUNet) to progressively aggregate hybrid features as well as to better trade-off the reconstruction accuracy and efficiency. Furthermore, we introduce a Refinement module to further refine edge and texture details with global information. Experimental results on numerous benchmarks validate that the proposed TCSR achieves superior performance with fewer parameters and less computational overhead than state-of-the-art lightweight methods.
Funding Information
  • National Natural Science Foundation of China (61901183)
  • Natural Science Foundation of Fujian Province (2023J01140)
  • Natural Science Foundation of Fujian Provincial Science and Technology Department (2021H6037, 2023H6034)
  • Key Project of Quanzhou Science and Technology Plan (2021C008R, 2023C007R)
  • 2022 National College Student Innovation and Entrepreneurship Training Program Project (202210399010)
  • Sixth Batch of Quanzhou City’s Introduction of High-Level Talent Team Projects No2

This publication has 49 references indexed in Scilit: