Xiayan Zhang

Remote sensing satellite images with high temporal and high spatial resolution play a critical role in earth science applications. However, it is difficult for a single satellite to obtain such images due to technical and cost constraints. Therefore, spatiotemporal image fusion based on deep learning has received extensive attention in recent years. This article proposes a multicooperative deep convolutional neural network (MCDNet) for spatiotemporal satellite image fusion. This method is a new multinetwork model in which multiple networks work together to reconstruct the predicted image. The multinetwork model consists of a super-resolution network, a difference reconstruction network, and a collaborative training network. First, the super-resolution network uses the combination of a novel multiscale mechanism and dilated convolutions to make full use of the spectral information of the coarse image and upgrade it to a transitional image that matches the fine image. The difference reconstruction network uses structural relevance to complete the reconstruction of the fine difference image. The collaborative training network extracts the hidden information from the fine image and uses the time relevance to restrict the training of the difference reconstruction network. Finally, the fine difference image and the known fine image are combined to complete the image fusion. The new compound loss function can help multinetwork models better complete cooperative training. Through experiments on two datasets and comparison with existing fusion algorithms, the subjective and objective results prove that MCDNet can effectively reconstruct higher-quality prediction images.

In this paper, we propose a new spatiotemporal fusion method based on a convolutional neural network to which we added attention and multiscale mechanisms (AMNet). Different from the previous spatiotemporal fusion methods, the residual image obtained by subtracting moderate resolution imaging spectroradiometer (MODIS) images at two times is directly used to train the network, and two special structures of multiscale mechanism and attention mechanism are used to increase the accuracy of fusion. Our proposed method uses one pair of images to achieve spatiotemporal fusion. The work is mainly divided into three steps. The first step is to extract feature maps of two types of images at different scales and fuse them separately. The second step is to use the attention mechanism to focus on the important information in the feature maps. And the third step is to reconstruct the image. We used two classical datasets for the experiment, and compared our experimental results with the other three state-of-the-art spatiotemporal fusion methods. The results of our proposed method have richer spatial details and more accurate prediction of temporal changes.