AFF-Net: Adaptive Frequency-Domain Fusion in a Dual-Stream Transformer With Wavelet-Assisted Alignment for Remote Sensing Image Segmentation
- 20 May 2026
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Geoscience and Remote Sensing Letters
- Vol. 23 (1545598X) , 6011305
Abstract
Remote sensing semantic segmentation is essential for geoscientific analysis. Recently, although multimodal image fusion has achieved notable improvements over single-image approaches, existing methods still suffer from insufficient utilization of the observed information, especially the alignment and fusion of multimodal details information. To address above issues, a novel adaptive frequency-domain fusion network (AFF-Net) is proposed in this letter to achieve effective fusion of visible images and DSM data through a Frequency-Domain Fusion (FDF) module, and enhance the recognition capability of target boundaries via a Semantic-Guided Detail Alignment (SDA) strategy. Specifically, the FDF module conducts an adaptive frequency-wise alignment and fusion scheme between visible images and DSM by decomposing them into low and high frequency subbands, enabling the model to capture more fine-grained complementary information across various frequency domains, rather than coarse-grained fusion in the spatial domain. The SDA strategy further restores high-frequency boundary details while suppressing noise, enabling precise object delineation. AFF-Net was evaluated on the ISPRS Vaihingen and Potsdam datasets, achieving mIoU scores of 84.87%, 87.11%, as well as mF1 scores of 91.61% and 92.93%, respectively. These results show better performances over state-of-the-art methods and verify the effectiveness and superiority of the proposed method. The code is available at https://github.com/lanmao-create/AFF-Net.Keywords
Funding Information
- Program for University Key Laboratory of Guangdong Province (26IRTSTHN003)
- State Key Laboratory of Drug Research, Chinese Academy of Sciences (SKLSD2025-HNZZ-01)
This publication has 20 references indexed in Scilit:
- BFRNet: Bimodal Fusion and Rectification Network for Remote Sensing Semantic Segmentation2024
- MMF-CLIP: An Image-Text Multimodal Semantic Segmentation Method for Remote Sensing Images2024
- UNetFormer: A UNet-like transformer for efficient semantic segmentation of remote sensing urban scene imageryISPRS Journal of Photogrammetry and Remote Sensing, 2022
- A2-FPN for semantic segmentation of fine-resolution remotely sensed imagesInternational Journal of Remote Sensing, 2022
- A Novel Transformer Based Semantic Segmentation Scheme for Fine-Resolution Remote Sensing ImagesIEEE Geoscience and Remote Sensing Letters, 2022
- ABCNet: Attentive bilateral contextual network for efficient semantic segmentation of Fine-Resolution remotely sensed imageryISPRS Journal of Photogrammetry and Remote Sensing, 2021
- Probabilistic Modeling of Semantic Ambiguity for Scene Graph Generation2021
- Beyond RGB: Very high resolution urban remote sensing with multimodal deep networksISPRS Journal of Photogrammetry and Remote Sensing, 2018
- Focal Loss for Dense Object Detection2017
- A theory for multiresolution signal decomposition: the wavelet representationIEEE Transactions on Pattern Analysis and Machine Intelligence, 1989