Xiaodong Zhang

Multimodal remote sensing technology significantly enhances the understanding of surface semantics by integrating heterogeneous data such as optical images, Synthetic Aperture Radar (SAR), and Digital Surface Models (DSM). However, in practical applications, the missing of modality data (e.g., optical or DSM) is a common and severe challenge, which leads to performance decline in traditional multimodal fusion models. Existing methods for addressing missing modalities still face limitations, including feature collapse and overly generalized recovered features. To address these issues, we propose \textbf{STARS} (\textbf{S}hared-specific \textbf{T}ranslation and \textbf{A}lignment for missing-modality \textbf{R}emote \textbf{S}ensing), a robust semantic segmentation framework for incomplete multimodal inputs. STARS is built on two key designs. First, we introduce an asymmetric alignment mechanism with bidirectional translation and stop-gradient, which effectively prevents feature collapse and reduces sensitivity to hyperparameters. Second, we propose a Pixel-level Semantic sampling Alignment (PSA) strategy that combines class-balanced pixel sampling with cross-modality semantic alignment loss, to mitigate alignment failures caused by severe class imbalance and improve minority-class recognition.

The surface fragmentation of karst landscapes leads to a high degree of coupling between fire scar site boundaries and topographic relief. However, the applicability of spatio-temporal data fusion methods for fire scar extraction in such geomorphological areas remains systematically unevaluated. This study developed a spatial–temporal adaptive fusion model integrating Landsat 30-m data with MODIS daily observations to generate continuous high-precision dNBR datasets. Using a typical karst fire region in Guizhou and Yunnan, China, as a case study, we validated the method’s effectiveness for fire trace extraction in fragmented landscapes. The proposed fusion technique addresses cloud cover limitations in humid climates by constructing continuous NBR time series, enabling precise fire boundary delineation and severity quantification. We comparatively implemented multiple fusion approaches (FSDAF, STARFM, and STDFA) and evaluated their performance through both spectral (RMSE, AD, and PSNR) and spatial (Edge, LBP, and SSIM) metrics. Key findings include the following: (1) FSDAF outperformed other methods in spectral consistency and spatial adaptation, particularly for heterogeneous mountainous terrain with fragmented vegetation. (2) Comparative experiments demonstrated that pre-calculating vegetation indices before temporal fusion (Strategy I) produced superior results to post-fusion calculation (Strategy II). Moreover, in our karst landscape study area, our proposed Hybrid Strategy selection framework can dynamically optimize the fusion process of multi-source satellite data, which is significantly better than a single fusion strategy. (3) The dNBR-based extraction achieved 90.00% producer accuracy, 69.23% user accuracy, and a Kappa coefficient of 0.718 when validated against field data. This study advances fire monitoring in karst regions by significantly improving both the spatio-temporal resolution and accuracy of burn scar detection compared to conventional approaches. The framework provides a viable solution for fire impact assessment in topographically complex landscapes under cloudy conditions.

Focusing on problems of blurred detection boundary, small target miss detection, and more pseudo changes in high-resolution remote sensing image change detection, a change detection algorithm based on Siamese neural networks is proposed. Siam-FAUnet can implement end-to-end change detection tasks. Firstly, the improved VGG16 is utilized as an encoder to extract the image features. Secondly, the atrous spatial pyramid pooling module is used to increase the receptive field of the model to make full use of the global information of the image and obtain the multi-scale contextual information of the image. The flow alignment module is used to fuse the low-level features in the encoder to the decoder and solve the problem of semantic misalignment caused by the direct concatenation of features when the features are fused, so as to obtain the change region of the image. The experiments are trained and tested using publicly available CDD and SZTAKI datasets. The results show that the evaluation metrics of the Siam-FAUnet model are improved compared to the baseline model, in which the F1-score is improved by 4.00% on the CDD and by 7.32% and 2.62% on the sub-datasets of SZTAKI (SZADA and TISZADOB), respectively; compared to other state-of-the-art methods, the Siam-FAUnet model has improved in both evaluation metrics, indicating that the model has a good detection performance.

In recent years, remote sensing techniques such as satellite and drone-based imaging have been used to monitor Pine Wilt Disease (PWD), a widespread forest disease that causes the death of pine species. Researchers have explored the use of remote sensing imagery and deep learning algorithms to improve the accuracy of PWD detection at the single-tree level. This study introduces a novel framework for PWD detection that combines high-resolution RGB drone imagery with free-access Sentinel-2 satellite multi-spectral imagery. The proposed approach includes an PWD-infected tree detection model named YOLOv5-PWD and an effective data augmentation method. To evaluate the proposed framework, we collected data and created a dataset in Xianning City, China, consisting of object detection samples of infected trees at middle and late stages of PWD. Experimental results indicate that the YOLOv5-PWD detection model achieved 1.2% higher mAP compared to the original YOLOv5 model and a further improvement of 1.9% mAP was observed after applying our dataset augmentation method, which demonstrates the effectiveness and potential of the proposed framework for PWD detection.

Semantic segmentation is a fundamental task in remote sensing image analysis (RSIA). Fully convolutional networks (FCNs) have achieved state-of-the-art performance in the task of semantic segmentation of natural scene images. However, due to distinctive differences between natural scene images and remotely-sensed (RS) images, FCN-based semantic segmentation methods from the field of computer vision cannot achieve promising performances on RS images without modifications. In previous work, we proposed an RS image semantic segmentation framework SDFCNv1, combined with a majority voting postprocessing method. Nevertheless, it still has some drawbacks, such as small receptive field and large number of parameters. In this paper, we propose an improved semantic segmentation framework SDFCNv2 based on SDFCNv1, to conduct optimal semantic segmentation on RS images. We first construct a novel FCN model with hybrid basic convolutional (HBC) blocks and spatial-channel-fusion squeeze-and-excitation (SCFSE) modules, which occupies a larger receptive field and fewer network model parameters. We also put forward a data augmentation method based on spectral-specific stochastic-gamma-transform-based (SSSGT-based) during the model training process to improve generalizability of our model. Besides, we design a mask-weighted voting decision fusion postprocessing algorithm for image segmentation on overlarge RS images. We conducted several comparative experiments on two public datasets and a real surveying and mapping dataset. Extensive experimental results demonstrate that compared with the SDFCNv1 framework, our SDFCNv2 framework can increase the mIoU metric by up to 5.22% while only using about half of parameters.

Satellite video single object tracking has attracted wide attention. The development of remote sensing platforms for earth observation technologies makes it increasingly convenient to acquire high-resolution satellite videos, which greatly accelerates ground target tracking. However, overlarge images with small object size, high similarity among multiple moving targets, and poor distinguishability between the objects and the background make this task most challenging. To solve these problems, a deep Siamese network (DSN) incorporating an interframe difference centroid inertia motion (ID-CIM) model is proposed in this paper. In object tracking tasks, the DSN inherently includes a template branch and a search branch; it extracts the features from these two branches and employs a Siamese region proposal network to obtain the position of the target in the search branch. The ID-CIM mechanism was proposed to alleviate model drift. These two modules build the ID-DSN framework and mutually reinforce the final tracking results. In addition, we also adopted existing object detection datasets for remotely sensed images to generate training datasets suitable for satellite video single object tracking. Ablation experiments were performed on six high-resolution satellite videos acquired from the International Space Station and “Jilin-1” satellites. We compared the proposed ID-DSN results with other 11 state-of-the-art trackers, including different networks and backbones. The comparison results show that our ID-DSN obtained a precision criterion of 0.927 and a success criterion of 0.694 with a frames per second (FPS) value of 32.117 implemented on a single NVIDIA GTX1070Ti GPU.

Previously, generative adversarial networks (GAN) have been widely applied on super resolution reconstruction (SRR) methods, which turn low-resolution (LR) images into high-resolution (HR) ones. However, as these methods recover high frequency information with what they observed from the other images, they tend to produce artifacts when processing unfamiliar images. Optical satellite remote sensing images are of a far more complicated scene than natural images. Therefore, applying the previous networks on remote sensing images, especially mid-resolution ones, leads to unstable convergence and thus unpleasing artifacts. In this paper, we propose Enlighten-GAN for SRR tasks on large-size optical mid-resolution remote sensing images. Specifically, we design the enlighten blocks to induce network converging to a reliable point, and bring the Self-Supervised Hierarchical Perceptual Loss to attain performance improvement overpassing the other loss functions. Furthermore, limited by memory, large-scale images need to be cropped into patches to get through the network separately. To merge the reconstructed patches into a whole, we employ the internal inconsistency loss and cropping-and-clipping strategy, to avoid the seam line. Experiment results certify that Enlighten-GAN outperforms the state-of-the-art methods in terms of gradient similarity metric (GSM) on mid-resolution Sentinel-2 remote sensing images.

Auto-extraction of convective clouds is of great significance. Convective clouds often bring heavy rain, strong winds, and other disastrous weather. Early warning of convection can effectively reduce loss. Using remote sensing images, we can get large-scale cloud information, which provides many effective methods for convective clouds detection. In this letter, we proposed a novel method to extract convective clouds. We introduce a novel deep network using only 1 x1 convolution (3ONet) to extract the spectral characteristics. We then combine a 3ONet with the symmetrical dense-shortcut deep fully convolutional networks (SDFCNs) with a double-stream fully convolutional network to extract convective clouds. In the experiment, we used 12,000 Himawari-8 satellite image patches to verify the proposed framework. Experimental results with 0.5882 mean intersection over union (mIOU) pointed out the proposed method can extract convective clouds effectively.