Similarity Weight Learning: A New Spatial and Temporal Satellite Image Fusion Framework
Authors:
- 21 March 2022
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Geoscience and Remote Sensing
- Vol. 60 (01962892) , 1-17
Abstract
Spatiotemporal fusion is a topical framework for solving the mutual restricted problem between the spatial and temporal resolution of satellite images. We pioneer an approach to replace similarity measurement steps in spatiotemporal fusion algorithms with convolutional neural networks (CNNs), building a bridge between weight function-based models and the learning-based models. Specifically, we propose a nonlocal form that separates the relational computation part from the value representation part, and construct the CNN-based similarity weight learning block for learning normalized weights. The block can be inserted into spatial and temporal adaptive reflectance fusion model (STARFM) to replace the manually designed weight calculation rules common in weight function-based methods, or into the CNN model StfNet to better utilize neighboring high-resolution images. The trained model outputs a high-resolution prediction from each base date image pair. The final result is a combination of the two predictions. In this regard, we propose the standard deviation-based weights to combine two prediction results. Four experiments are performed on Landsat–Moderate-resolution Imaging Spectroradiometer (MODIS) image pairs to determine the following: 1) the performance of the model at the target training date; 2) the generalization of the model in the target training time period; and 3) the generalization of the model at different dates and different geographical locations, each considering the different cases of giving one and two pairs of known images. Experimental results demonstrate the superiority of the similarity weight learning block and standard deviation-based weights. Among them, STARFM with the similarity weight learning block exhibits strong generalization, which testifies to the practical value of our model.Keywords
Funding Information
- National Natural Science Foundation of China (42071250)
This publication has 43 references indexed in Scilit:
- A spatial and temporal reflectance fusion model considering sensor observation differencesInternational Journal of Remote Sensing, 2013
- An enhanced spatial and temporal adaptive reflectance fusion model for complex heterogeneous regionsRemote Sensing of Environment, 2010
- A new data fusion model for high spatial- and temporal-resolution mapping of forest disturbance based on Landsat and MODISRemote Sensing of Environment, 2009
- Monitoring mangrove forest dynamics of the Sundarbans in Bangladesh and India using multi-temporal satellite data from 1973 to 2000Estuarine, Coastal and Shelf Science, 2007
- On the use of MODIS EVI to assess gross primary productivity of North American ecosystemsJournal of Geophysical Research: Biogeosciences, 2006
- Land-cover change detection using multi-temporal MODIS NDVI dataRemote Sensing of Environment, 2006
- Cloud cover in Landsat observations of the Brazilian AmazonInternational Journal of Remote Sensing, 2001
- Unmixing-based multisensor multiresolution image fusionIEEE Transactions on Geoscience and Remote Sensing, 1999
- How unique are spectral signatures?Remote Sensing of Environment, 1994
- A Computer Movie Simulating Urban Growth in the Detroit RegionEconomic Geography, 1970