Zilong Li

Remote sensing video (RSV) provides continuous, high spatiotemporal earth observations that are increasingly important for environmental monitoring, disaster response, infrastructure inspection and urban management. Despite this potential, operational use of video streams is hindered by very large data volumes, heterogeneous acquisition platforms, inconsistent preprocessing practices, and the absence of standardized formats that deliver data ready for immediate analysis. These shortcomings force repeated low-level computation, complicate semantic extraction, and limit reproducibility and cross-sensor integration. This manuscript presents a principled multi-level analysis-ready data (ARD) model for remote sensing video, named VideoARD, along with VideoCube, a spatiotemporal management and query infrastructure that implements and operationalizes the model. VideoARD formalizes semantic abstraction at scene, object, and event levels and defines minimum and optimal readiness configurations for each level. The proposed pipeline applies stabilization, georeferencing, key frame selection, object detection, trajectory tracking, event inference, and entity materialization. VideoCube places the resulting entities into a five-dimensional structure indexed by spatial, temporal, product, quality, and semantic dimension, and supports earth observation OLAP-style operations to enable efficient slicing, aggregation, and drill down. Benchmark experiments and three application studies, covering vessel speed monitoring, wildfire detection, and near-real-time three-dimensional reconstruction, quantify system performance and operational utility. Results show that the proposed approach achieves multi-gigabyte-per-second ingestion under parallel feeds, sub-second scene retrieval for typical queries, and second-scale trajectory reconstruction for short tracks. Case studies demonstrate faster alert generation, improved detection consistency, and substantial reductions in preprocessing and manual selection work compared with on-demand baselines. The principal trade-off is an upfront cost for materialization and storage that becomes economical when queries are repeated or entities are reused. The contribution of this work lies in extending the analysis-ready data concept from static imagery to continuous video streams and in delivering a practical, scalable architecture that links semantic abstraction to high-performance spatiotemporal management, thereby improving responsiveness, reproducibility, and cross-sensor analysis for Earth observation.

Earth observation (EO) video analysis emerges as an attractive solution for various applications including moving vehicle tracking, disaster monitoring, and environmental change detection. Conventional study focus on single-source video analysis, which limits observations to narrow temporal window. By extending from single-source to multi-source approaches, cross-patform EO video analysis offers a promising solution for continuous object tracking over extends time periods. It should be noted that multi-source EO videos refer to videos captured by different platforms or sensors at different locations or times. However, cross EO video tracking presents several unexplored challenges, including spatio-temporal and color misalignments between different sources, the inherent difficulties of detecting small objects from aerial perspectives, and the high error rates encountered in cross EO video object reidentification. Accordingly, in this paper, we propose a new framework targeting for Cross EO Video Tiny Object Tracking (CVTOT) that comprises three core components: (1) a multi-source EO alignment module for spatio-temporal and color registration; (2) a modified YOLO framework for tiny EO object detection; and (3) an enhanced motion matrix with a geo-awareness matching strategy to realize robust object reidentification across videos. Additionaly, we introduce the CVOT benchmark (CVOT21 and CVOT230), the first dataset tailored for cross EO video tiny object tracking, with explicit annotations of object handover(enter/exit) events and cross EO video identity links. Extensive experiments on this benchmark demonstrate that the proposed CVTOT pipeline significantly enhances tracking consistency (e.g., IDF1 21.4%↑) and retrieval accuracy (e.g., CVC 74.3%↑) under realistic multi-source domain shifts.

Earth Observation (EO) videos are undergoing rapid expansion due to the swift advancements in aerial, spaceborne, and ground remote sensing technologies enabling the continuous capture of imagery of the Earth's surface. Compared to traditional image-based EO data, EO videos offer persistent Earth observations, rendering a promising observation resource across diverse applications, including climate monitoring and hazard assessment. The continuous observation capability introduces a challenge to the community, i.e., how to effectively manage and fully harness the value of the substantial volume of EO videos. In this paper, we propose a novel approach leveraging a spatiotemporal data cube with EO video management to facilitate analysis. It suggests an Analysis Ready Data (ARD) for EO videos, termed as Analysis Ready Video Data (ARVD), which is incorporated into an Earth Video Cube (EVC). The ARD includes semantics at frame, object/trajectory, and event levels. The paper presents the cube data organization and query processing for EO videos. A prototype system is implemented to demonstrate the applicability of the approach.

Recent development of remote sensing technologies brings Earth observation (EO) videos as a new type of satellite imagery in a spatial data infrastructure (SDI). Although SDI has been applied successfully for traditional EO data management, EO videos in SDI still face some problems including incompatible video formats of different product systems and the registration of videos with GIS (Geographic Information System) maps. The paper presents an SDI approach on how to manage EO videos, taking Jilin-1 satellite videos as an example. It suggests an open standards based approach for acquiring, storing, accessing, and processing EO videos. Some key technologies including video storage, video object extraction, and alignment of video and ground maps are highlighted.

Thin clouds seriously affect the availability of optical remote sensing images, especially in visible bands. Short-wave infrared (SWIR) bands are less influenced by thin clouds, but usually have lower spatial resolution than visible (Vis) bands in high spatial resolution remote sensing images (e.g., in Sentinel-2A/B, CBERS04, ZY-1 02D and HJ-1B satellites). Most cloud removal methods do not take advantage of the spectral information available in SWIR bands, which are less affected by clouds, to restore the background information tainted by thin clouds in Vis bands. In this paper, we propose CR-MSS, a novel deep learning-based thin cloud removal method that takes the SWIR and vegetation red edge (VRE) bands as inputs in addition to visible/near infrared (Vis/NIR) bands, in order to improve cloud removal in Sentinel-2 visible bands. Contrary to some traditional and deep learning-based cloud removal methods, which use manually designed rescaling algorithm to handle bands at different resolutions, CR-MSS uses convolutional layers to automatically process bands at different resolution. CR-MSS has two input/output branches that are designed to process Vis/NIR and VRE/SWIR, respectively. Firstly, Vis/NIR cloudy bands are down-sampled by a convolutional layer to low spatial resolution features, which are then concatenated with the corresponding features extracted from VRE/SWIR bands. Secondly, the concatenated features are put into a fusion tunnel to down-sample and fuse the spectral information from Vis/NIR and VRE/SWIR bands. Third, a decomposition tunnel is designed to up-sample and decompose the fused features. Finally, a transpose convolutional layer is used to up-sample the feature maps to the resolution of input Vis/NIR bands. CR-MSS was trained on 28 real Sentinel-2A image pairs over the globe, and tested separately on eight real cloud image pairs and eight simulated cloud image pairs. The average SSIM values (Structural Similarity Index Measurement) for CR-MSS results on Vis/NIR bands over all testing images were 0.69, 0.71, 0.77, and 0.81, respectively, which was on average 1.74% higher than the best baseline method. The visual results on real Sentinel-2 images demonstrate that CR-MSS can produce more realistic cloud and cloud shadow removal results than baseline methods.

Cross-modal hashing (CMH) models are introduced to significantly reduce the cost of large-scale cross-modal data retrieval systems. In many real-world applications, however, data of new categories arrive continuously, which requires the model has good extensibility. That is the model should be updated to accommodate data of new categories but still retain good performance for the old categories with minimum computation cost. Unfortunately, existing CMH methods fail to satisfy the extensibility requirements. In this work, we propose a novel extensible cross-modal hashing (ECMH) to enable highly efficient and low-cost model extension. Our proposed ECMH has several desired features: 1) it has good forward compatibility, so there is no need to update old hash codes; 2) the ECMH model is extended to support new data categories using only new data by a well-designed ``weak constraint incremental learning'' algorithm, which saves up to 91\% time cost comparing with retraining the model with both new and old data; 3) the extended model achieves high precision and recall on both old and new tasks. Our extensive experiments show the effectiveness of our design.