Semantic Segmentation of Remote Sensing Images With Self-Supervised Semantic-Aware Inpainting
- 6 October 2022
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Geoscience and Remote Sensing Letters
- Vol. 19 (1545598X) , 1-5
Abstract
Semantic segmentation of remote sensing imageries plays a crucial role in resource exploration, urban planning, weather forecasting, etc. For this task, deep learning-based methods have shown significant achievement, typically trained with large-scale labeled data. However, these methods often suffer the performance deterioration facing limited labeled data in real-world applications. To address this problem, a novel self-supervised semantic segmentation framework is proposed for remote sensing imageries with limited labeled data. Specifically, image inpainting is acted as pixel-level pretext task for learning dense feature representations suitable for semantic segmentation. Furthermore, rather than trivially leveraging the conventional random inpainting strategy, a novel adversarial training scheme is proposed to drive the pretext task to adaptively mask and restore salient local regions. The adversarial training scheme consists of instructor network and inpainting network, the instructor network increasingly predicts meaningful salient regions as erased regions, and meanwhile the inpainting network seeks for restoring the corrupted image as pretext task to learn its intrinsic representation. Moreover, the structural similarity (SSIM) is applied as a patch-level loss function for semantic segmentation considering that remote sensing images are highly structured. The experimental results on the International Society for Photogrammetry and Remote Sensing (ISPRS) Potsdam dataset demonstrate that our method outperforms state-of-the-art self-supervised methods and the ImageNet pre-training methods. The source code is available at https://github.com/JasmineBJTU/self-supervised_RSSSKeywords
Funding Information
- Fundamental Research Funds for the Central Universities (2022JBQY007)
- National Natural Science Foundation of China (62006017, 62276019, U2034211)
This publication has 16 references indexed in Scilit:
- Semantic Segmentation of Remote Sensing Images With Self-Supervised Multitask Representation LearningIEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2021
- A Refined Pyramid Scene Parsing Network for Polarimetric SAR Image Semantic Segmentation in Agricultural AreasIEEE Geoscience and Remote Sensing Letters, 2021
- Momentum Contrast for Unsupervised Visual Representation Learning2020
- Dilated Residual Networks2017
- Unsupervised Learning of Visual Representations by Solving Jigsaw Puzzles2016
- Colorful Image Colorization2016
- Context Encoders: Feature Learning by Inpainting2016
- Deep Residual Learning for Image Recognition2016
- U-Net: Convolutional Networks for Biomedical Image Segmentation2015
- Image quality assessment: from error visibility to structural similarityIEEE Transactions on Image Processing, 2004