In-Domain Self-Supervised Learning Improves Remote Sensing Image Scene Classification
- 16 January 2024
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Geoscience and Remote Sensing Letters
- Vol. 21 (1545598X) , 1-5
Abstract
We investigate the utility of in-domain self-supervised pretraining of vision models in the analysis of remote sensing imagery. Self-supervised learning (SSL) has emerged as a promising approach for remote sensing image classification due to its ability to exploit large amounts of unlabeled data. Unlike traditional supervised learning, SSL aims to learn representations of data without the need for explicit labels. This is achieved by formulating auxiliary tasks that can be used for pretraining models before fine-tuning them on a given downstream task. A common approach in practice to SSL pretraining is utilizing standard pretraining datasets, such as ImageNet. While relevant, such a general approach can have a suboptimal influence on the downstream performance of models, especially on tasks from challenging domains such as remote sensing. In this letter, we analyze the effectiveness of SSL pretraining by employing the image bidirectional encoder representations from transformers (BERT) pretraining with online tokenizer (iBOT) framework coupled with Vision transformers (ViTs) trained on million aerial image dataset (Million-AID), a large and unlabeled remote sensing dataset. We present a comprehensive study of different self-supervised pretraining strategies and evaluate their effect across 14 downstream datasets with diverse properties. Our results demonstrate that leveraging large in-domain datasets for self-supervised pretraining consistently leads to improved predictive downstream performance, compared to the standard approaches found in practice.Keywords
All Related Versions
Funding Information
- European Space Agency (ESA) through the activity AiTLAS—Artificial Intelligence Toolbox for Earth Observation awarded to Bias Variance Labs, d.o.o. (ESA RFP/3-16371/19/I-NB)
This publication has 10 references indexed in Scilit:
- Self-Supervised Intra-Modal and Cross-Modal Contrastive Learning for Point Cloud UnderstandingIEEE Transactions on Multimedia, 2023
- AiTLAS: Artificial Intelligence Toolbox for Earth ObservationRemote Sensing, 2023
- Current trends in deep learning for Earth Observation: An open-source benchmark arena for image classificationISPRS Journal of Photogrammetry and Remote Sensing, 2023
- Advancing Plain Vision Transformer Toward Remote Sensing Foundation ModelIEEE Transactions on Geoscience and Remote Sensing, 2022
- Self-Supervised Learning in Remote Sensing: A reviewIEEE Geoscience and Remote Sensing Magazine, 2022
- Masked Autoencoders Are Scalable Vision Learners2022
- Seasonal Contrast: Unsupervised Pre-Training from Uncurated Remote Sensing Data2021
- Emerging Properties in Self-Supervised Vision Transformers2021
- On Creating Benchmark Dataset for Aerial Image Interpretation: Reviews, Guidances, and Million-AIDIEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2021
- Grad-CAM: Visual Explanations from Deep Networks via Gradient-Based Localization2017