Shumeng He

Change detection identifies dynamic changes in surface cover and feature status by comparing remote sensing images at different points in time, which is of wide application value in the fields of disaster early warning, urban management and ecological monitoring. Mainstream datasets are dominated by long‐term datasets; to support short‐term change detection, we collected a new dataset, HNU‐CD, which contains some small and hard‐to‐identify change regions. A time correlation network (TCNet) is also proposed to address these challenges. First, foreground information is enhanced by interactively modelling foreground relations, while background noise is smoothed. Secondly, the temporal correlation between bit‐time images is utilised to refine the feature representation and minimise false alarms due to irrelevant changes. Finally, a U‐Net inspired architecture is adapted for dense upsampling to preserve details. TCNet demonstrates excellent performance on both the HNU‐CD (Hainan University change detection dataset) dataset and three widely used public datasets, indicating that its generalisation capabilities have been enhanced. The ablation experiments provide a good demonstration of the ability to reduce the impact caused by pseudo‐variation through temporal correlation modelling.

Due to the inherent inter-class similarity and class imbalance of remote sensing images, it is difficult to obtain effective results in single-source semantic segmentation. We consider applying multi-modal data to the task of the semantic segmentation of HSR (high spatial resolution) remote sensing images, and obtain richer semantic information by data fusion to improve the accuracy and efficiency of segmentation. However, it is still a great challenge to discover how to achieve efficient and useful information complementarity based on multi-modal remote sensing image semantic segmentation, so we have to seriously examine the numerous models. Transformer has made remarkable progress in decreasing model complexity and improving scalability and training efficiency in computer vision tasks. Therefore, we introduce Transformer into multi-modal semantic segmentation. In order to cope with the issue that the Transformer model requires a large amount of computing resources, we propose a model, MFTransNet, which combines a CNN (convolutional neural network) and Transformer to realize a lightweight multi-modal semantic segmentation structure. To do this, a small convolutional network is first used for performing preliminary feature extraction. Subsequently, these features are sent to the multi-head feature fusion module to achieve adaptive feature fusion. Finally, the features of different scales are integrated together through a multi-scale decoder. The experimental results demonstrate that MFTransNet achieves the best balance among segmentation accuracy, memory-usage efficiency and inference speed.

High spatial resolution (HSR) remote sensing images have a wide range of application prospects in the fields of urban planning, agricultural planning and military training. Therefore, the research on the semantic segmentation of remote sensing images becomes extremely important. However, large data volume and the complex background of HSR remote sensing images put great pressure on the algorithm efficiency. Although the pressure on the GPU can be relieved by down-sampling the image or cropping it into small patches for separate processing, the loss of local details or global contextual information can lead to limited segmentation accuracy. In this study, we propose a multi-field context fusion network (MCFNet), which can preserve both global and local information efficiently. The method consists of three modules: a backbone network, a patch selection module (PSM), and a multi-field context fusion module (FM). Specifically, we propose a confidence-based local selection criterion in the PSM, which adaptively selects local locations in the image that are poorly segmented. Subsequently, the FM dynamically aggregates the semantic information of multiple visual fields centered on that local location to enhance the segmentation of these local locations. Since MCFNet only performs segmentation enhancement on local locations in an image, it can improve segmentation accuracy without consuming excessive GPU memory. We implement our method on two high spatial resolution remote sensing image datasets, DeepGlobe and Potsdam, and compare the proposed method with state-of-the-art methods. The results show that the MCFNet method achieves the best balance in terms of segmentation accuracy, memory efficiency, and inference speed.

Summary: Semantic segmentation of remote sensing images plays a significant role in many applications such as urban planning and ecological protection, but its semantic segmentation suffers from large intra‐category variation and large differences in the scale of objects, so it is prone to misclassification. To cope with this issue, an embedded channel's categorical attention module (ECCA) is proposed to extract contextual information from the perspective of categories, and a channel attention module is embedded in it to achieve multiple contextual information extraction. Combined with the remote sensing atrous spatial pyramid pooling module (RSASPP), which is composed of atrous convolution with different expansion rates, feature fusion of objects at different scales is achieved. The refinement module (RM) is added for boundary refinement to achieve finer segmentation. Experiments are conducted on the WHDLD dataset to prove the effectiveness of the method.