Domain Adaptive Semantic Segmentation of Remote Sensing Images via Self-Training-Based Dual-Level Data Augmentation
Open Access
- 17 October 2024
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing
- Vol. 17 (19391404) , 19713-19729
Abstract
Semantic segmentation models experience a significant performance degradation due to domain shifts between the source and target domains. This issue is particularly prevalent in remote sensing imagery, where a semantic segmentation model trained on images from one satellite is tested on images from another. Previous research has often overlooked the role of data augmentation in enhancing a model's adaptability to target domains. In contrast, this paper proposes a novel self-training framework that incorporates data augmentation at both the input and feature levels, yielding excellent results. Specifically, we introduce a regularized online self-training framework that effectively addresses the challenges of overconfidence and class imbalance inherent in self-training. Based on this framework, we implement two robust data augmentation strategies at the input and feature levels to facilitate the learning of cross-domain invariant knowledge. At the input level, we employ a large-scale domain mixing strategy, termed Multi-Domain Mixing (MDM), to enhance the model's generalization capability. At the feature level, we introduce Masked Feature Augmentation (MFA), a masking-based perturbation technique applied to the semantic features of the target domain. This approach enhances the consistency of teacher-student network predictions in the target domain feature space, thereby improving the robustness of the model's recognition of target domain features. The integration of the proposed self-training framework with dual-level data augmentation culminates in our innovative STDA method. Extensive experimental results on the ISPRS semantic segmentation benchmark demonstrate that STDA outperforms existing stateof-the-art methods, showcasing its effectiveness.Keywords
Funding Information
- National Key Laboratory of Science and Technology on Space-Born Intelligent Information Processing (TJ-01-22-01)
- National Natural Science Foundation of China (62301046)
- De_velopment Program of China (2023YFC3605900)
This publication has 59 references indexed in Scilit:
- HD-Net: High-resolution decoupled network for building footprint extraction via deeply supervised body and boundary decompositionISPRS Journal of Photogrammetry and Remote Sensing, 2024
- HRDA: Context-Aware High-Resolution Domain-Adaptive Semantic Segmentation2022
- Prototypical Contrast Adaptation for Domain Adaptive Semantic Segmentation2022
- Learning deep semantic segmentation network under multiple weakly-supervised constraints for cross-domain remote sensing image semantic segmentationISPRS Journal of Photogrammetry and Remote Sensing, 2021
- Instance Adaptive Self-training for Unsupervised Domain Adaptation2020
- ResUNet-a: A deep learning framework for semantic segmentation of remotely sensed dataISPRS Journal of Photogrammetry and Remote Sensing, 2020
- Algorithms for semantic segmentation of multispectral remote sensing imagery using deep learningISPRS Journal of Photogrammetry and Remote Sensing, 2018
- Unsupervised Domain Adaptation for Semantic Segmentation via Class-Balanced Self-training2018
- Domain Transfer Through Deep Activation Matching2018
- U-Net: Convolutional Networks for Biomedical Image Segmentation2015