FALSE: False Negative Samples Aware Contrastive Learning for Semantic Segmentation of High-Resolution Remote Sensing Image
- 17 November 2022
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Geoscience and Remote Sensing Letters
- Vol. 19 (1545598X) , 1-5
Abstract
Self-supervised contrastive learning (SSCL) is a potential learning paradigm for learning remote sensing image (RSI)-invariant features through the label-free method. The existing SSCL of RSI is built based on constructing positive and negative sample pairs. However, due to the richness of RSI ground objects and the complexity of the RSI contextual semantics, the same RSI patches have the coexistence and imbalance of positive and negative samples, which causing the SSCL pushing negative samples far away while pushing positive samples far away, and vice versa. We call this the sample confounding issue (SCI). To solve this problem, we propose a False negAtive sampLes aware contraStive lEarning model (FALSE) for the semantic segmentation of high-resolution RSIs. Since the SSCL pretraining is unsupervised, the lack of definable criteria for false negative sample (FNS) leads to theoretical undecidability, we designed two steps to implement the FNS approximation determination: coarse determination of FNS and precise calibration of FNS. We achieve coarse determination of FNS by the FNS self-determination (FNSD) strategy and achieve calibration of FNS by the FNS confidence calibration (FNCC) loss function. Experimental results on three RSI semantic segmentation datasets demonstrated that the FALSE effectively improves the accuracy of the downstream RSI semantic segmentation task compared with the current three models, which represent three different types of SSCL models. The mean Intersection-over-Union on ISPRS Potsdam dataset is improved by 0.7% on average; on CVPR DGLC dataset is improved by 12.28% on average; and on Xiangtan dataset this is improved by 1.17% on average. This indicates that the SSCL model has the ability to self-differentiate FNS and that the FALSE effectively mitigates the SCI in self-supervised contrastive learning.Keywords
All Related Versions
Funding Information
- National Natural Science Foundation of China (41871364, 41861048)
- High Performance Computing Platform of Central South University
This publication has 17 references indexed in Scilit:
- Deep Learning for Land Use and Land Cover Classification Based on Hyperspectral and Multispectral Earth Observation Data: A ReviewRemote Sensing, 2020
- Remote Sensing Image Scene Classification Meets Deep Learning: Challenges, Methods, Benchmarks, and OpportunitiesIEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2020
- Momentum Contrast for Unsupervised Visual Representation Learning2020
- Deep learning in environmental remote sensing: Achievements and challengesRemote Sensing of Environment, 2020
- SCAttNet: Semantic Segmentation Network With Spatial and Channel Attention Mechanism for High-Resolution Remote Sensing ImagesIEEE Geoscience and Remote Sensing Letters, 2020
- Object detection in optical remote sensing images: A survey and a new benchmarkISPRS Journal of Photogrammetry and Remote Sensing, 2020
- High-Resolution Remote Sensing Imagery Classification of Imbalanced Data Using Multistage Sampling Method and Deep Neural NetworksRemote Sensing, 2019
- Deep Clustering for Unsupervised Learning of Visual Features2018
- DeepGlobe 2018: A Challenge to Parse the Earth through Satellite Images2018
- THE ISPRS BENCHMARK ON URBAN OBJECT CLASSIFICATION AND 3D BUILDING RECONSTRUCTIONISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences, 2012