Thick Cloud Removal Under Land Cover Changes Using Multisource Satellite Imagery and a Spatiotemporal Attention Network
- 11 January 2023
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Geoscience and Remote Sensing
- Vol. 61 (01962892) , 1-18
Abstract
Remote sensing satellites provide observations of the earth’s surface, which are crucial data for applications and analyses in several fields, including agriculture, environmental protection, and sustainable development. However, the wide and frequent occurrence of clouds highly undermines the quality and availability of usable optical data, particularly low-temporal-resolution data. Although deep learning techniques have facilitated recent progress in cloud removal algorithms, thick cloud removal under changing land cover remains challenging. In this study, we propose a framework to remove thick clouds, thin clouds, and cloud shadow from Sentinel-2 images. The framework integrates the spatial detail in a Sentinel-2 reference image and the coarse spectral pattern in a near-target-date Sentinel-3 image as spatiotemporal guidance to generate missing data with land cover change information in a cloudy Sentinel-2 image. The reconstruction is performed using a spatiotemporal attention network (STAN) that adopts the self-attention mechanism, residual learning, and high-pass features to enhance feature extraction from the multisource data. The experimental results show that STAN outperforms ResUnet, CRN, STS-CNN, and DSen2-CR in terms of multiple quantitative metrics and visual characteristics. The comparative experiment proves the integration of Sentinel-3 data improves the cloud removal performance, especially in areas with distinctive and heterogeneous land cover changes under large-scale cloud cover. The experimental results also indicate high generalizability of STAN when the Sentinel-3 image is far from the target date, when transferring features to cloud removal for new images, and even with limited training data that simulates severe cloud cover.Keywords
Funding Information
- National Key Research and Development Program of China (2019YFC1510400, 2022YFB3903700)
- Research Grants Council of Hong Kong (AoE/E-603/18)
- National Natural Science Foundation of China (42271439)
- Second Tibetan Plateau Scientific Expedition and Report (2019QZKK0308)
This publication has 59 references indexed in Scilit:
- Sparse-based reconstruction of missing information in remote sensing images from spectral/temporal complementary informationISPRS Journal of Photogrammetry and Remote Sensing, 2015
- Improvement and expansion of the Fmask algorithm: cloud, cloud shadow, and snow detection for Landsats 4–7, 8, and Sentinel 2 imagesRemote Sensing of Environment, 2015
- Cloud Removal From Optical Satellite Imagery With SAR Imagery Using Sparse RepresentationIEEE Geoscience and Remote Sensing Letters, 2015
- An effective thin cloud removal procedure for visible remote sensing imagesISPRS Journal of Photogrammetry and Remote Sensing, 2014
- Cloud removal for remotely sensed images by similar pixel replacement guided with a spatio-temporal MRF modelISPRS Journal of Photogrammetry and Remote Sensing, 2014
- Recovering Quantitative Remote Sensing Products Contaminated by Thick Clouds and Shadows Using Multitemporal Dictionary LearningIEEE Transactions on Geoscience and Remote Sensing, 2014
- Sentinel-2: ESA's Optical High-Resolution Mission for GMES Operational ServicesRemote Sensing of Environment, 2012
- The Global Monitoring for Environment and Security (GMES) Sentinel-3 missionRemote Sensing of Environment, 2012
- Monitoring 25 years of land cover change dynamics in Africa: A sample based remote sensing approachApplied Geography, 2009
- The availability of cloud-free Landsat ETM+ data over the conterminous United States and globallyRemote Sensing of Environment, 2008