Laplacian-Former: Overcoming the Limitations of Vision Transformers in Local Texture Detection
Authors:
- 1 October 2023
- book chapter
- Published by Springer Nature
- Vol. 14222, 736-746
Abstract
Vision Transformer (ViT) models have demonstrated a breakthrough in a wide range of computer vision tasks. However, compared to the Convolutional Neural Network (CNN) models, it has been observed that the ViT models struggle to capture high-frequency components of images, which can limit their ability to detect local textures and edge information. As abnormalities in human tissue, such as tumors and lesions, may greatly vary in structure, texture, and shape, high-frequency information such as texture is crucial for effective semantic segmentation tasks. To address this limitation in ViT models, we propose a new technique, Laplacian-Former, that enhances the self-attention map by adaptively re-calibrating the frequency information in a Laplacian pyramid. More specifically, our proposed method utilizes a dual attention mechanism via efficient attention and frequency attention while the efficient attention mechanism reduces the complexity of self-attention to linear while producing the same output, selectively intensifying the contribution of shape and texture features. Furthermore, we introduce a novel efficient enhancement multi-scale bridge that effectively transfers spatial information from the encoder to the decoder while preserving the fundamental features. We demonstrate the efficacy of Laplacian-former on multi-organ and skin lesion segmentation tasks with +1.87% and +0.76% dice scores compared to SOTA approaches, respectively. Our implementation is publically available at GitHub. Keywords Deep Learning Texture Segmentation Laplacian TransformerKeywords
All Related Versions
This publication has 16 references indexed in Scilit:
- TransDeepLab: Convolution-Free Transformer-Based DeepLab v3+ for Medical Image Segmentation2022
- LeVit-UNet: Make Faster Encoders with Transformer for Biomedical Image SegmentationSSRN Electronic Journal, 2022
- FAT-Net: Feature adaptive transformers for automated skin lesion segmentationMedical Image Analysis, 2022
- UNETR: Transformers for 3D Medical Image Segmentation2022
- Swin Transformer: Hierarchical Vision Transformer using Shifted Windows2021
- CoTr: Efficiently Bridging CNN and Transformer for 3D Medical Image Segmentation2021
- TransBTS: Multimodal Brain Tumor Segmentation Using Transformer2021
- On the Texture Bias for Few-Shot CNN Segmentation2021
- Attention gated networks: Learning to leverage salient regions in medical imagesMedical Image Analysis, 2019
- U-Net: Convolutional Networks for Biomedical Image Segmentation2015