Lei Lin

Optical satellite time series data play a crucial role in monitoring vegetation dynamics and land surface changes. However, persistent cloud cover often leads to missing data, particularly during critical phenological stages, which significantly diminishes data quality and hinders downstream applications. To address this issue, we present CosmDiff (Conditional Optical-SAR Multi-temporal Diffusion), a novel framework for reconstructing optical satellite time series by integrating multimodal, multi-temporal optical and Synthetic Aperture Radar (SAR) data using conditional diffusion models. In CosmDiff, the reconstruction task is formulated as a multivariate time series imputation problem, where missing values are modeled as conditionally dependent on both cloudfree optical observations and synergic SAR time series. The framework incorporates a Transformer-based network within the diffusion process, introducing a novel dimensional decomposition attention mechanism that fuses optical-SAR time series across both temporal and feature dimensions. This mechanism enables the dynamic extraction of essential and complementary features from both modalities. Additionally, linearly interpolated optical time series are used as auxiliary inputs to further guide the imputation process. Experimental results on Sentinel-1/-2 datasets demonstrate that CosmDiff consistently outperforms both traditional interpolation methods and advanced deep learning approaches, achieving a 3.8% reduction in mean absolute error (MAE) and a 6.8% improvement in spectral angle mapper (SAM) compared to competing methods. Furthermore, CosmDiff provides comprehensive uncertainty estimates for its predictions, which are particularly valuable for decision-making applications.

Satellite image time series (SITS) are widely used for land cover mapping and vegetation monitoring. Despite the success of deep learning methods in SITS classification, their performance strongly depends on large labeled datasets. Data augmentation is a cost-effective strategy to prevent deep learning models from overfitting with limited labeled data, but its effectiveness for SITS has yet to be thoroughly explored. This paper provides an empirical study of eleven alternative augmentation techniques for pixel-wise satellite time series, including Noise Injection, Scaling, Mixup, Weighted DBA (Dynamic Time Warping Barycentric Averaging), Temporal Dropout, Window Slicing, Temporal Shift, Time Warping, Interpolation Resampling, Amplitude Jittering, and Phase Jittering. Notably, Interpolation Resampling was introduced to handle irregularly-sampled satellite time series, enhancing model robustness to data incompleteness and spatiotemporal heterogeneity. We evaluated the performance gains of different augmentation techniques and their combinations on both same-year and cross-year test data under varying conditions (sequence length, sample size, time period, paramter setting) and assessed their processing speeds. Based on the results, we summarized the conditions under which different augmentation techniques are effective and provided a systematic analysis of their performance. Our study offers practical guidance for data augmentation in various SITS classification applications. The source code is available at https://github.com/linlei1214/SITS-Aug.

Satellite image time series (SITS) classification is a major research topic in remote sensing and is relevant for a wide range of applications. Deep learning approaches have been commonly employed for SITS classification and have provided state-of-the-art performance. However, deep learning methods suffer from overfitting when labeled data is scarce. To address this problem, we propose a novel self-supervised pre-training scheme to initialize a Transformer-based network by utilizing large-scale unlabeled data. In detail, the model is asked to predict randomly contaminated observations given an entire time series of a pixel. The main idea of our proposal is to leverage the inherent temporal structure of satellite time series to learn general-purpose spectral-temporal representations related to land cover semantics. Once pre-training is completed, the pre-trained network can be further adapted to various SITS classification tasks by fine-tuning all the model parameters on small-scale task-related labeled data. In this way, the general knowledge and representations about SITS can be transferred to a label-scarce task, thereby improving the generalization performance of the model as well as reducing the risk of overfitting. Comprehensive experiments have been carried out on three benchmark datasets over large study areas. Experimental results demonstrate the effectiveness of the proposed pre-training scheme, leading to substantial improvments in classification accuracy using Transformer, one-dimensional convolutional neural network (CNN-1D), and Bidirectional Long Short-Term Memory (Bi-LSTM) network. The code and the pre-trained model will be available at https://github.com/linlei1214/SITS-BERT upon publication.

Accurate prediction of future observations based on past data is the key to near real-time disturbance detection using satellite image time series (SITS). To overcome the limitations of existing methods, we present an attention-based LSTM Encoder-Decoder model in which the historical time series of a pixel is encoded with a bidirectional LSTM encoder while the future time series is produced by another LSTM decoder. An attention mechanism is integrated into the encoder-decoder model to align the input time series with the output time series and to dynamically choose the most relevant contextual information while forecasting. Based on the proposed model, we develop a framework for near real-time disturbance detection and verify its effectiveness in the case of burned area mapping. The prediction accuracy of the proposed model is evaluated using Moderate Resolution Imaging Spectroradiometer (MODIS) time series and compared with state-of-the-art models. Experimental results show that our model achieves the best results in terms of lower prediction error and higher model fitness. We also evaluate the disturbance detection ability of the proposed framework. The proposed approach improves the detection rate of disturbances while suppressing false alarms, and increases the temporal accuracy. We suggest that the proposed methods provide new tools for enhancing current early warning systems in real time.

Large amounts of farmland loss caused by urban expansion has been a severe global environmental problem. Therefore, monitoring urban encroachment upon farmland is a global issue. In this study, we propose a novel framework for modelling and monitoring the conversion of cultivated land to built-up land using a satellite image time series (SITS). The land-cover change process is modelled by a two-level hierarchical hidden semi-Markov model, which is composed of two Markov chains with hierarchical relationships. The upper chain represents annual land-cover dynamics, and the lower chain encodes the vegetation phenological patterns of each land-cover type. This kind of architecture enables us to represent the multilevel semantic information of SITS at different time scales. Specifically, intra-annual series reflect phenological differences and inter-annual series reflect land-cover dynamics. In this way, we can take advantage of the temporal information contained in the entire time series as well as the prior knowledge of land cover conversion to identify where and when changes occur. As a case study, we applied the proposed method for mapping annual, long-term urban-induced farmland loss from Moderate Resolution Imaging Spectroradiometer (MODIS) Normalized Difference Vegetation Index (NDVI) time series in the Jing-Jin-Tang district, China from 2001 to 2010. The accuracy assessment showed that the proposed method was accurate for detecting conversions from cultivated land to built-up land, with the overall accuracy of 97.72% in the spatial domain and the temporal accuracy of 74.60%. The experimental results demonstrated the superiority of the proposed method in comparison with other state-of-the-art algorithms. In addition, the spatial-temporal patterns of urban expansion revealed in this study are consistent with the findings of previous studies, which also confirms the effectiveness of the proposed method.

In recent years, the tensor factorization model has been used to model complicated feature interactions involving multiple aspects, such as the user, publisher, and advertiser, for response prediction in real-time bidding. Among numerous challenges, data sparsity and cold start problems always bother researchers, particularly for ad conversion rate prediction. Such problems in prediction become difficult if only one or several types of information are considered. All types of heterogeneous information must be simultaneously integrated to address these problems. This paper proposes an availability solution for integrating heterogeneous information in the tensor factorization model to efficiently alleviate data sparsity and cold start problems. It proposes different integration strategies and implementation methods for various types of information depending on their property, category, structure, form, and function. This solution efficiently alleviates data sparsity and cold start problems, and enhances the prediction reliability and precision for the tensor factorization model in real-time bidding systems. Finally, this solution achieves a significant improvement in response prediction compared to baselines methods on the selection datasets.

Due to its capacity for temporal and spatial coverage, remote sensing has emerged as a powerful tool for mapping inundation. Many methods have been applied effectively in remote sensing flood analysis. Generally, supervised methods can achieve better precision than unsupervised. However, human intervention makes its results subjective and difficult to obtain automatically, which is important for disaster response. In this work, we propose a novel procedure combining spatiotemporal context learning method and Modest AdaBoost classifier, which aims to extract inundation in an automatic and accurate way. First, the context model was built with images to calculate the confidence value of each pixel, which represents the probability of the pixel remaining unchanged. Then, the pixels with the highest probabilities, which we define as ‘permanent pixels’, were used as samples to train the Modest AdaBoost classifier. By applying the strong classifier to the target scene, an inundation map can be obtained. The proposed procedure is validated using two flood cases with different sensors, HJ-1A CCD and GF-4 PMS. Qualitative and quantitative evaluation results showed that the proposed procedure can achieve accurate and robust mapping results.

Fire detection based on multi-temporal remote sensing data is an active research field. However, multi-temporal detection processes are usually complicated because of the spatial and temporal variability of remote sensing imagery. This paper presents a spatio-temporal model (STM) based forest fire detection method that uses multiple images of the inspected scene. In STM, the strong correlation between an inspected pixel and its neighboring pixels is considered, which can mitigate adverse impacts of spatial heterogeneity on background intensity predictions. The integration of spatial contextual information and temporal information makes it a more robust model for anomaly detection. The proposed algorithm was applied to a forest fire in 2009 in the Yinanhe forest, Heilongjiang province, China, using two-month HJ-1B infrared camera sensor (IRS) images. A comparison of detection results demonstrate that the proposed algorithm described in this paper are useful to represent the spatio-temporal information contained in multi-temporal remotely sensed data, and the STM detection method can be used to obtain a higher detection accuracy than the optimized contextual algorithm.

In this paper, we propose a novel method to continuously monitor land cover change using satellite image time series, which can extract comprehensive change information including change time, location, and “from-to” information. This method is based on a hidden Markov model (HMM) trained for each land cover class. Assuming a pixel’s initial class has been obtained, likelihoods of the corresponding model are calculated on incoming time series extracted with a temporal sliding window. By observing the likelihood change over the windows, land cover change can be precisely detected from the dramatic drop of likelihood. The established HMMs are then used for identifying the land cover class after the change. As a case study, the proposed method is applied to monitoring urban encroachment onto farmland in Beijing using 10-year MODIS time series from 2001 to 2010. The performance is evaluated on a validation set for different model structures and thresholds. Compared with other change detection methods, the proposed method shows superior change detection accuracy. In addition, it is also more computationally efficient.