Zhang Li

Advances in "track-anything" models have significantly improved fine-grained video understanding by simultaneously handling multiple video segmentation and tracking tasks. However, existing models often struggle with robust and efficient temporal propagation. To address these challenges, we propose the Sparse Spatio-Temporal Propagation (SSTP) method, which achieves robust and efficient unified video segmentation by selectively leveraging key spatio-temporal features in videos. Specifically, we design a dynamic 3D spatio-temporal convolution to aggregate global multi-frame spatio-temporal information into memory frames during memory construction. Additionally, we introduce a spatio-temporal aggregation reading strategy to efficiently aggregate the relevant spatio-temporal features from multiple memory frames during memory retrieval. By combining SSTP with an image segmentation foundation model, such as the segment anything model, our method effectively addresses multiple data-scarce video segmentation tasks. Our experimental results demonstrate state-of-the-art performance on five video segmentation tasks across eleven datasets, outperforming both task-specific and unified methods. Notably, SSTP exhibits strong robustness in handling sparse, low-frame-rate videos, making it well-suited for real-world applications.

In this paper, we present a Unified Framework for cross-domain Space drone Pose Estimation (UF-SPE), addressing the simulation-to-reality gap that limits the deployment of deep learning models in real space missions. The proposed UF-SPE framework integrates offline domain generalization with online unsupervised domain adaptation. During offline training, the model relies exclusively on synthetic images. It employs advanced augmentation techniques and a multi-task architecture equipped with Domain Shifting Uncertainty modules to improve the learning of domain-invariant features. In the online phase, normalization layers are fine-tuned using unlabeled real-world imagery via entropy minimization, allowing for the system to adapt to target domain distributions without manual labels. Experiments on the SPEED+ benchmark demonstrate that the UF-SPE achieves competitive accuracy with just 12.9 M parameters, outperforming the comparable lightweight baseline method by 37.5% in pose estimation accuracy. The results validate the framework’s efficacy and efficiency for robust cross-domain space drone pose estimation, indicating promise for applications such as on-orbit servicing, debris removal, and autonomous rendezvous.

Optical-SAR image matching is a fundamental task for remote sensing applications. While existing methods perform well on some popular datasets such as SEN1-2 and WHU-SEN-City, their generalizability across diverse data sources such as satellites, spatial resolutions, and scenes remains insufficiently investigated, hindering the practical implementation of optical-SAR matching in various downstream tasks. Thus, 3MOS, the first multi-source, multi-resolution, and multi-scene optical-SAR dataset, was proposed in our study to address this gap. This dataset consists of 113k optical-SAR image pairs, with the SAR data collected from five satellites and resolutions ranging from 3.5 m to 12.5 m, further categorized into eight scenes, such as urban, rural, and plains through a simple but practical classification strategy. Based on this dataset, the performance of optical-SAR matching methods was evaluated through the data with diverse characteristics. Additionally, extensive experiments were conducted, and the following two findings were obtained. 1) None of the state-of-the-art methods achieved consistently superior performance across different sources, resolutions, and scenes, specifying significant generalization challenges for diverse downstream task data. 2) Training data distribution significantly impacted the matching performance of deep-learning models, highlighting the domain adaptation challenge in optical-SAR image matching. Furthermore, the practical utility of the dataset was comprehensively validated through multimodal change detection experiments, demonstrating its substantial value for a wide range of downstream applications.

Current research on change detection has made significant progress on large-scale landscapes and buildings. However, there is a lack of exploration into the status changes of multiclass and time-sensitive objects across different temporal of remote sensing images (RSIs). To bridge this gap, we first introduce a task termed multiclass object change detection (MCOCD) and then construct a dedicated dataset dubbed aircraft change detection (ACD). Furthermore, we propose a postdetection framework to address this task. In the framework, we first feed bitemporal RSIs into an object detector to obtain the bounding boxes (BBOXs) of predefined classes. Subsequently, we utilize the intersection over union (IoU)-based distance to ascertain changes. Nervelessly, due to the dense arrangement of objects in RSIs, directly using IoU-based distance to determine changes results in one-to-many or many-to-one matching problems. To address this issue, we propose an optimal transport (OT) module to compute the global optimal matching of distance matrices, which are bidirectional augmented with dustbin nodes. Finally, the detected objects that are matched with the dustbin nodes being regarded as the changed ones. Extensive experiments demonstrate the effectiveness of our methods.

Tracking and measuring targets using a variety of sensors mounted on UAVs is an effective means to quickly and accurately locate the target. This paper proposes a fusion localization method based on ridge estimation, combining the advantages of rich scene information from sequential imagery with the high precision of laser ranging to enhance localization accuracy. Under limited conditions such as long distances, small intersection angles, and large inclination angles, the column vectors of the design matrix have serious multicollinearity when using the least squares estimation algorithm. The multicollinearity will lead to ill-conditioned problems, resulting in significant instability and low robustness. Ridge estimation is introduced to mitigate the serious multicollinearity under the condition of limited observation. Experimental results demonstrate that our method achieves higher localization accuracy compared to ground localization algorithms based on single information. Moreover, the introduction of ridge estimation effectively enhances the robustness, particularly under limited observation conditions.

With the advancement of remote sensing technologies, multimodal image matching has become increasingly important. Multimodal remote sensing images exhibit significant differences in radiometric and geometric properties due to their distinct imaging principles and being contaminated by various types of noise. Therefore, multimodal image matching remains a challenging problem. This paper proposes a novel descriptor called Shape-Adaptive Modality Independent Region Descriptor (SA-MIRD) to achieve robust matching across modalities by leveraging local structural self-similarities within the images. The proposed method first models the noise of each modality and uses a Local Polynomial Approximation-Intersection of Confidence Interval (LPA-ICI) to fit the local texture patterns. It then constructs the descriptor based on regional gradients extracted from local structural masks. Experimental results demonstrate the effectiveness of our approach in extracting structural features from multimodal images with good noise robustness, especially in regions with limited texture, such as mountainous and island areas. The proposed descriptor achieves high accuracy and robustness in multimodal image matching.

Estimating the pose (position and orientation) of a known but uncooperative satellite from a monocular image is a promising technique in space missions. Existing methods use deep neural networks and yield significant accuracy improvements over traditional ones. However, orbital applications require not only accuracy and real-time performance, but also the ability to know when and where the algorithm will fail. We develop a keypointset based method to achieve a better balance between accuracy, efficiency, and reliability. To achieve real-time performance, we first adopt a lightweight neural network to predict the semantic keypoints in the monocular image and then estimate the pose by solving the Perspective-n-Point (PnP) problem. To address the drop in pose estimation accuracy caused by the significant reduction in parameters, we extend the basic keypoint-set based approach via aleatoric uncertainty modeling, considering both spatial (coordinate regression) and semantic (index classification) uncertainties. Specifically, the spatial uncertainty modeling involves predicting Gaussian distribution parameters for the coordinates of each element. We then incorporate a weight for each keypoint when estimating the pose by solving the PnP problem and significantly improve the pose estimation accuracy as a result. We also explore semantic uncertainty and add a selfassessment mechanism using the mean entropy of keypoint logits to identify incorrect pose estimates. Extensive experiments on the SPEED dataset highlights the flexibility and effectiveness of our method.