Semantic Segmentation in Aerial Imagery Using Multi-level Contrastive Learning with Local Consistency
Authors:
- 1 January 2023
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 3787-3796
Abstract
Semantic segmentation in large-scale aerial images is an extremely challenging task. On one hand, the limited ground truth, as compared to the vast area the images cover, greatly hinders the development of supervised representation learning. On the other hand, the large footprint from remote sensing raises new challenges for semantic segmentation. In addition, the complex and ever changing image acquisition conditions further complicate the problem where domain shifting commonly occurs. In this paper, we exploit self-supervised contrastive learning (CL) methodologies for semantic segmentation in aerial imagery. In addition to performing CL at the feature level as most practices do, we add another level of contrastive learning, at the semantic level, taking advantage of the segmentation output from the downstream task. Further, we embed local mutual information in the semantic-level CL to enforce local consistency. This has largely enhanced the representation power at each pixel and improved the generalization capacity of the trained model. We refer to the proposed approach as multi-level contrastive learning with local consistency (mCL-LC). The experimental results on different benchmarks indicate that the proposed mCL-LC exhibits superior performance as compared to other state-of-the-art contrastive learning frameworks for the semantic segmentation task. mCL-LC also carries better generalization capacity especially when domain shifting exists.Keywords
Funding Information
- Office of the Director of National Intelligence
This publication has 33 references indexed in Scilit:
- Assisting Flood Disaster Response with Earth Observation Data and Products: A Critical AssessmentRemote Sensing, 2018
- Learning Features by Watching Objects Move2017
- Colorization as a Proxy Task for Visual Understanding2017
- Pyramid Scene Parsing Network2017
- DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFsIEEE Transactions on Pattern Analysis and Machine Intelligence, 2017
- The Cityscapes Dataset for Semantic Urban Scene Understanding2016
- Multimodal Registration via Mutual Information Incorporating Geometric and Spatial ContextIEEE Transactions on Image Processing, 2014
- The challenges of image segmentation in big remotely sensed imagery dataAnnals of GIS, 2014
- Microsoft COCO: Common Objects in Context2014
- Image registration methods: a surveyImage and Vision Computing, 2003