MB-TaylorFormer: Multi-branch Efficient Transformer Expanded by Taylor Formula for Image Dehazing
Authors:
Top Cited Papers
- 1 October 2023
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 12756-12767
Abstract
In recent years, Transformer networks are beginning to replace pure convolutional neural networks (CNNs) in the field of computer vision due to their global receptive field and adaptability to input. However, the quadratic computational complexity of softmax-attention limits the wide application in image dehazing task, especially for high-resolution images. To address this issue, we propose a new Transformer variant, which applies the Taylor expansion to approximate the softmax-attention and achieves linear computational complexity. A multi-scale attention refinement module is proposed as a complement to correct the error of the Taylor expansion. Furthermore, we introduce a multi-branch architecture with multi-scale patch embedding to the proposed Transformer, which embeds features by overlapping deformable convolution of different scales. The design of multi-scale patch embedding is based on three key ideas: 1) various sizes of the receptive field; 2) multi-level semantic information; 3) flexible shapes of the receptive field. Our model, named Multi-branch Transformer expanded by Taylor formula (MB-TaylorFormer), can em-bed coarse to fine features more flexibly at the patch embedding stage and capture long-distance pixel interactions with limited computational cost. Experimental results on several dehazing benchmarks show that MB-TaylorFormer achieves state-of-the-art (SOTA) performance with a light computational burden. The source code and pre-trained models are available at https://github.com/FVL2020/ICCV-2023-MB-TaylorFormer.Keywords
All Related Versions
Funding Information
- National Natural Science Foundation of China
This publication has 72 references indexed in Scilit:
- MaxViT: Multi-axis Vision Transformer2022
- MSTFDN: Multi-scale transformer fusion dehazing networkApplied Intelligence, 2022
- Deep Image Deblurring: A SurveyInternational Journal of Computer Vision, 2022
- Beyond Monocular Deraining: Parallel Stereo Deraining Network Via Semantic PriorInternational Journal of Computer Vision, 2022
- Learning Enriched Features for Real Image Restoration and Enhancement2020
- End-to-End Object Detection with Transformers2020
- Physics-Based Feature Dehazing Networks2020
- Recurrent Squeeze-and-Excitation Context Aggregation Net for Single Image Deraining2018
- Single Image Dehazing via Multi-scale Convolutional Neural Networks2016
- U-Net: Convolutional Networks for Biomedical Image Segmentation2015