Auto-CM: Unsupervised Deep Learning for Satellite Imagery Composition and Cloud Masking Using Spatio-Temporal Dynamics
Open Access
- 26 June 2023
- journal article
- research article
- Published by Association for the Advancement of Artificial Intelligence (AAAI) in Proceedings of the AAAI Conference on Artificial Intelligence
- Vol. 37 (12) , 14575-14583
Abstract
Cloud masking is both a fundamental and a critical task in the vast majority of Earth observation problems across social sectors, including agriculture, energy, water, etc. The sheer volume of satellite imagery to be processed has fast-climbed to a scale (e.g., >10 PBs/year) that is prohibitive for manual processing. Meanwhile, generating reliable cloud masks and image composite is increasingly challenging due to the continued distribution-shifts in the imagery collected by existing sensors and the ever-growing variety of sensors and platforms. Moreover, labeled samples are scarce and geographically limited compared to the needs in real large-scale applications. In related work, traditional remote sensing methods are often physics-based and rely on special spectral signatures from multi- or hyper-spectral bands, which are often not available in data collected by many -- and especially more recent -- high-resolution platforms. Machine learning and deep learning based methods, on the other hand, often require large volumes of up-to-date training data to be reliable and generalizable over space. We propose an autonomous image composition and masking (Auto-CM) framework to learn to solve the fundamental tasks in a label-free manner, by leveraging different dynamics of events in both geographic domains and time-series. Our experiments show that Auto-CM outperforms existing methods on a wide-range of data with different satellite platforms, geographic regions and bands.Keywords
This publication has 16 references indexed in Scilit:
- Replication across space and time must be weak in the social and environmental sciencesProceedings of the National Academy of Sciences, 2021
- Comparison of cloud detection algorithms for Sentinel-2 imageryScience of Remote Sensing, 2020
- Cloud detection for Landsat imagery by combining the random forest and superpixels extracted via energy-driven sampling segmentation approachesRemote Sensing of Environment, 2020
- Deep learning for multi-modal classification of cloud, shadow and land cover scenes in PlanetScope and Sentinel-2 imageryISPRS Journal of Photogrammetry and Remote Sensing, 2019
- Fmask 4.0: Improved cloud and cloud shadow detection in Landsats 4–8 and Sentinel-2 imageryRemote Sensing of Environment, 2019
- Multi-sensor cloud and cloud shadow segmentation with a convolutional neural networkRemote Sensing of Environment, 2019
- Validation of the LaSRC Cloud Detection Algorithm for Landsat 8 ImagesIEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2019
- Cloud detection algorithm comparison and validation for operational Landsat data productsRemote Sensing of Environment, 2017
- A cloud image detection method based on SVM vector machineNeurocomputing, 2015
- A multi-temporal method for cloud detection, applied to FORMOSAT-2, VENµS, LANDSAT and SENTINEL-2 imagesRemote Sensing of Environment, 2010