DASRSNet: Multitask Domain Adaptation for Super-Resolution-Aided Semantic Segmentation of Remote Sensing Images
- 26 December 2022
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Geoscience and Remote Sensing
- Vol. 61 (01962892) , 1-18
Abstract
Unsupervised domain adaptation (UDA) has become an important technique for cross-domain semantic segmentation (SS) in remote sensing community and obtained remarkable results. However, when transferring from high-resolution (HR) remote sensing images to low-resolution (LR) images, the existing UDA frameworks always fail to segment the LR target images, especially for small objects (e.g., cars), due to the severe spatial resolution shift problem. In this paper, to improve the segmentation ability of UDA models for LR target images and small objects, we propose a novel multi-task domain adaptation network (DASRSNet) for SS of remote sensing images with the aid of super-resolution (SR). The proposed DASRSNet contains domain adaptation for semantic segmentation (DASS) branch, domain adaptation for super-resolution (DASR) branch and feature affinity (FA) module. Specifically, the DASS and DASR branches share the same encoder to extract the domain-invariant features for the target and source domains, and these two branches utilize different decoders and discriminators to conduct cross-domain SS task and SR task, which align the domain shift in output space and image space respectively. Finally, FA module, which involves the proposed FA loss, is applied to enhance the affinity of SS features and SR features for both source and target domains. The experimental results on the cross-city aerial datasets demonstrate the effectiveness and superiority of our DASRSNet against the recent UDA models.Keywords
Funding Information
- National Natural Science Foundation of China (61825205)
- Key Research and Development Program of Zhejiang Province, China (2021C01017)
This publication has 48 references indexed in Scilit:
- Learning Aerial Image Segmentation From Online MapsIEEE Transactions on Geoscience and Remote Sensing, 2017
- DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFsIEEE Transactions on Pattern Analysis and Machine Intelligence, 2017
- Accelerating the Super-Resolution Convolutional Neural Network2016
- Domain Adaptation for the Classification of Remote Sensing Data: An Overview of Recent AdvancesIEEE Geoscience and Remote Sensing Magazine, 2016
- Accurate Image Super-Resolution Using Very Deep Convolutional Networks2016
- Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network2016
- Is image super-resolution helpful for other vision tasks?2016
- Image Super-Resolution Using Deep Convolutional NetworksIEEE Transactions on Pattern Analysis and Machine Intelligence, 2015
- What is a good evaluation measure for semantic segmentation?2013
- Cubic convolution interpolation for digital image processingIEEE Transactions on Acoustics, Speech, and Signal Processing, 1981