Spatiotemporal Fusion of Satellite Images via Very Deep Convolutional Networks
Open Access
- 18 November 2019
- journal article
- research article
- Published by MDPI AG in Remote Sensing
- Vol. 11 (22) , 2701
Abstract
Spatiotemporal fusion provides an effective way to fuse two types of remote sensing data featured by complementary spatial and temporal properties (typical representatives are Landsat and MODIS images) to generate fused data with both high spatial and temporal resolutions. This paper presents a very deep convolutional neural network (VDCN) based spatiotemporal fusion approach to effectively handle massive remote sensing data in practical applications. Compared with existing shallow learning methods, especially for the sparse representation based ones, the proposed VDCN-based model has the following merits: (1) explicitly correlating the MODIS and Landsat images by learning a non-linear mapping relationship; (2) automatically extracting effective image features; and (3) unifying the feature extraction, non-linear mapping, and image reconstruction into one optimization framework. In the training stage, we train a non-linear mapping between downsampled Landsat and MODIS data using VDCN, and then we train a multi-scale super-resolution (MSSR) VDCN between the original Landsat and downsampled Landsat data. The prediction procedure contains three layers, where each layer consists of a VDCN-based prediction and a fusion model. These layers achieve non-linear mapping from MODIS to downsampled Landsat data, the two-times SR of downsampled Landsat data, and the five-times SR of downsampled Landsat data, successively. Extensive evaluations are executed on two groups of commonly used Landsat–MODIS benchmark datasets. For the fusion results, the quantitative evaluations on all prediction dates and the visual effect on one key date demonstrate that the proposed approach achieves more accurate fusion results than sparse representation based methods.Keywords
This publication has 25 references indexed in Scilit:
- Image Super-Resolution Using Deep Convolutional NetworksIEEE Transactions on Pattern Analysis and Machine Intelligence, 2015
- Assessing the accuracy of blending Landsat–MODIS surface reflectances in two landscapes with contrasting spatial and temporal dynamics: A framework for algorithm selectionRemote Sensing of Environment, 2013
- Spatiotemporal Reflectance Fusion via Sparse RepresentationIEEE Transactions on Geoscience and Remote Sensing, 2012
- An enhanced spatial and temporal adaptive reflectance fusion model for complex heterogeneous regionsRemote Sensing of Environment, 2010
- Image Super-Resolution Via Sparse RepresentationIEEE Transactions on Image Processing, 2010
- A new data fusion model for high spatial- and temporal-resolution mapping of forest disturbance based on Landsat and MODISRemote Sensing of Environment, 2009
- The assessment of multi-sensor image fusion using wavelet transforms for mapping the Brazilian SavannaInternational Journal of Applied Earth Observation and Geoinformation, 2006
- Image quality assessment: from error visibility to structural similarityIEEE Transactions on Image Processing, 2004
- Gradient-based learning applied to document recognitionProceedings of the IEEE, 1998
- Backpropagation Applied to Handwritten Zip Code RecognitionNeural Computation, 1989