Two-stream spatiotemporal image fusion network based on difference transformation
- 1 July 2022
- journal article
- research article
- Published by SPIE-Intl Soc Optical Eng in Journal of Applied Remote Sensing
- Vol. 16 (03) , 038506-15
Abstract
For satellite imaging instruments, the tradeoff between spatial and temporal resolution leads to the spatial–temporal contradiction of image sequences. Spatiotemporal image fusion (STIF) provides a solution to generate images with both high-spatial and high-temporal resolutions, thus expanding the applications of existing satellite images. Most deep learning-based STIF methods throw the task to network as a whole and construct an end-to-end model without caring about the intermediate physical process. This leads to high complexity, less interpretability, and low accuracy of the fusion model. To address this problem, we propose a two-stream difference transformation spatiotemporal fusion (TSDTSF), which includes transformation and fusion streams. In the transformation stream, an image difference transformation module reduces the pixel distribution difference of images from different sensors with the same spatial resolution, and a feature difference transformation module improves the feature quality of low-resolution images. The fusion stream focuses on feature fusion and image reconstruction. The TSDTSF shows superior performance in accuracy, vision quality, and robustness. The experimental results show that TSDTSF achieves the effect of the average coefficient of determination (R2 = 0.7847) and the root mean square error (RMSE = 0.0266), which is better than the suboptimal method average (R2 = 0.7519) and (RMSE = 0.0289). The quantitative and qualitative experimental results on various datasets demonstrate our superiority over the state-of-the-art methods.Keywords
This publication has 37 references indexed in Scilit:
- A flexible spatiotemporal method for fusing satellite images with different resolutionsRemote Sensing of Environment, 2016
- An Error-Bound-Regularized Sparse Coding for Spatiotemporal Reflectance FusionIEEE Transactions on Geoscience and Remote Sensing, 2015
- Assessing the accuracy of blending Landsat–MODIS surface reflectances in two landscapes with contrasting spatial and temporal dynamics: A framework for algorithm selectionRemote Sensing of Environment, 2013
- A spatial and temporal reflectance fusion model considering sensor observation differencesInternational Journal of Remote Sensing, 2013
- Preparing Landsat Image Time Series (LITS) for Monitoring Changes in Vegetation Phenology in Queensland, AustraliaRemote Sensing, 2012
- Downscaling real-time vegetation dynamics by fusing multi-temporal MODIS and Landsat NDVI in topographically complex terrainRemote Sensing of Environment, 2011
- An enhanced spatial and temporal adaptive reflectance fusion model for complex heterogeneous regionsRemote Sensing of Environment, 2010
- A new data fusion model for high spatial- and temporal-resolution mapping of forest disturbance based on Landsat and MODISRemote Sensing of Environment, 2009
- Multiscale structural similarity for image quality assessment2004
- Unmixing-based multisensor multiresolution image fusionIEEE Transactions on Geoscience and Remote Sensing, 1999