DHT: Deformable Hybrid Transformer for Aerial Image Segmentation

Authors:
Publisher website
Google Scholar
Abstract
Due to the strong ability to model global information, the transformer-based methods have shown remarkable improvements in image segmentation tasks. However, the self-attention mechanism in the transformer is computationally expensive and relies on pre-trained parameters. Moreover, the transformer method is weak in modeling local information, which is unfavorable for accurately segmenting objects from high-resolution aerial images. To this end, an efficient deformable orientational self-attention (DoA) is proposed to simultaneously extract the global information and the local information. Besides, for parameter efficiency, we design a depthwise channel self-attention (DcA) to model the contextual information among channels. Combining with the DoA and DcA, we propose the deformable hybrid transformer (DHT) to perform high-quality object segmentation on aerial images. Experiments on ISPRS Potsdam dataset and WHU building dataset illustrate that the proposed DHT can not only achieve state-of-the-art (SOTA) results but also markedly reduce the dependence of the transformer on pre-trained parameters.
Funding Information
  • National Natural Science Foundation of China (62036007, 62176195, 62206034)
  • Special Project on Technological Innovation and Application Development (cstc2020jscx-dxwtB0032)
  • Chongqing Excellent Scientist Project (cstc2021ycjh-bgzxm0339)
  • Natural Science Foundation of Chongqing Municipality (cstc2021jcyj-msxmX0847)
  • Science and Technology Research Program of Chongqing Municipal Education Commission (KJQN202200644, KJQN202200618)

This publication has 14 references indexed in Scilit: