Xiaoxuan Liu

Low-altitude economy, fine-grained surveying, emergency response, and autonomous exploration are driving Unmanned Aerial Vehicles (UAVs) from passive data-acquisition platforms toward task-executing aerial agents. This capability transition requires UAVs to operate within complex, open 3D environments under six-degree-of-freedom (6DoF) motion, strict size, weight and power (SWaP) limits, partial observations, onboard computation constraints, and safety-critical action requirements. Therefore, the central scientific problem of UAV 3D scene understanding is how a UAV agent can construct, maintain, and use a spatiotemporally coherent and uncertainty-aware 3D scene state to support localization, planning, and safe interaction. Existing surveys mainly categorize the literature according to sensor types, application scenarios, or generic 3D representations, and thus provide limited analysis of how 3D scene understanding supports agent-capability evolution under embodied aerial constraints. To address this gap, we review UAV 3D scene understanding along an agent-capability evolution from offline interpretation to online understanding and predictive reasoning. This perspective highlights the underlying tensions between representation fidelity and onboard deployability, open-vocabulary semantic coverage and calibrated trustworthiness, post-flight static reconstruction and online scene-state maintenance, and predictive reasoning and safety-bounded decision support. Finally, we discuss open challenges in closed-loop data construction, trustworthy scene-state memory, collaborative fusion, sim-to-real transfer, and reliable onboard deployment.

As a key task in remote sensing analysis, semantic segmentation of remote sensing images (RSI) underpins many practical applications. Despite its importance, obtaining dense pixel-wise annotations remains labor-intensive and time-consuming. Unsupervised domain adaptation (UDA) offers a promising solution by utilizing knowledge from labeled source domains for unlabeled target domains, yet its effectiveness is often compromised by significant distribution shifts arising from variations in imaging conditions. To address this challenge, we propose a depth-aware adaptation network (DAAN), a novel two-branch network that explicitly leverages complementary depth information from a digital surface model (DSM) to enhance cross-domain remote sensing segmentation. Unlike conventional UDA methods that primarily focus on semantic features, DAAN incorporates depth data to build a more generalized feature space. This network introduces three key components: an adaptive feature aggregator (AFA) for progressive semantic-depth feature fusion, a gated prediction selection unit (GPSU) that selectively integrates predictions to mitigate the impact of noisy depth measurements, and misalignment-focused residual refinement (MFRR) module that emphasizes poorly aligned target regions during training. Experiments on the ISPRS and GAMUS datasets demonstrate the effectiveness of the proposed method. In particular, DAAN achieves an mIoU of 50.53% and an F1 score of 65.75% for cross-domain segmentation on ISPRS to GAMUS, outperforming models without depth information by 9.17% and 8.99%, respectively. These results demonstrate the advantage of integrating auxiliary geometric information to improve model generalization on unlabeled remote sensing datasets, contributing to higher mapping accuracy, more reliable automated analysis, and enhanced decision-making support.

The analysis of dynamic remote sensing scenes from unmanned aerial vehicles (UAVs) is shifting from reactive processing to proactive, predictive intelligence. Central to this evolution is video prediction—forecasting future imagery from past observations—which enables critical remote sensing applications like persistent environmental monitoring, occlusion-robust object tracking, and infrastructure anomaly detection under challenging aerial conditions. Yet, a systematic review of video prediction models tailored for the unique constraints of aerial remote sensing has been lacking. Existing taxonomies often obscure key design choices, especially for emerging operators like state-space models (SSMs). We address this gap by proposing a unified, multi-dimensional taxonomy with three orthogonal axes: (i) operator architecture; (ii) generative nature; and (iii) training/inference regime. Through this lens, we analyze recent methods, clarifying their trade-offs for deployment on UAV platforms that demand processing of high-resolution, long-horizon video streams under tight resource constraints. Our review assesses the utility of these models for key applications like proactive infrastructure inspection and wildlife tracking. We then identify open problems—from the scarcity of annotated aerial video data to evaluation beyond pixel-level metrics—and chart future directions. We highlight a convergence toward scalable dynamic world models for geospatial intelligence, which leverage physics-informed learning, multimodal fusion, and action-conditioning, powered by efficient operators like SSMs.

The goal of Few-Shot Segmentation (FSS) is to segment images of novel categories using few labeled examples. However, FSS tasks face challenges such as over-segmentation and lack of generalization issues. This paper addresses these challenges by employing a triple attention mechanism (TAM) and a hierarchical decoding transformer (HDT). Specifically, TAM is proposed to enhance the model’s ability to focus on spatial regions within query features that are semantically relevant to the target category. The HDT module then aggregates the enhanced query features with the support features in a decoupled manner, generating dense features with pixel-level semantic relevance, which improves the segmentation ability on novel classes. Additionally, considering that class-level labels inside image can provide weak supervision for the segmentation task, this paper introduces a contrastive language image pretraining (CLIP) based model to enhance the segmentation performance. The Grad-CAM mechanism is utilized to convert the class logit scores from CLIP into localization heatmaps, effectively leveraging the text label information to provide prior localization cues for the model. Extensive experiments conducted on the PASCAL-5ⁱ and COCO-20ⁱ datasets demonstrate state-of-the-art performance. The experimental results validate the effectiveness of the proposed method, significantly improving the generalization and segmentation performance of few-shot semantic segmentation models on novel categories.

Traditional Multiple Object Tracking (MOT) methods in satellite videos mostly follow the Detection-Based Tracking (DBT) framework. However, the DBT framework assumes that all objects are correctly recognized and localized by the detector. In practice, the low resolution of satellite videos, small objects, and complex backgrounds inevitably leads to a decline in detector performance. To alleviate the impact of detector degradation on track, we propose Coarse-Fine Tracker, a framework that integrates the MOT framework with the Tracking Any Point (TAP) method CoTracker for the first time, leveraging TAP’s persistent point correspondence modeling to compensate for detector failures. In our Coarse-Fine Tracker, we divide the satellite video into sub-videos. For one sub-video, we first use ByteTrack to track the outputs of the detector, referred to as coarse tracking, which involves the Kalman filter and box-level motion features. Given the small size of objects in satellite videos, we treat each object as a point to be tracked. We then use CoTracker to track the center point of each object, referred to as fine tracking, by calculating the appearance feature similarity between each point and its neighboring points. Finally, the Consensus Fusion Strategy eliminates mismatched detections in coarse tracking results by checking their geometric consistency against fine tracking results and recovers missed objects via linear interpolation or linear fitting. This method is validated on the VISO and SAT-MTB datasets. Experimental results in VISO show that the tracker achieves a multi-object tracking accuracy (MOTA) of 66.9, a multi-object tracking precision (MOTP) of 64.1, and an IDF1 score of 77.8, surpassing the detector-only baseline by 11.1% in MOTA while reducing ID switches by 139. Comparative experiments with ByteTrack demonstrate the robustness of our tracking method when the performance of the detector deteriorates.

Language-guided remote sensing image object localization uses intuitive natural language interactions to locate objects of interest within satellite or drone imagery, and has a wide range of practical applications. Early research on this task typically used discriminative models that relied on predefined task heads, limited to locating a single object and lacking flexibility. Recent years, multi-modal large language models based generative models leverage their language understanding ability to comprehend more complex language references and provide flexible outputs. However, these models tend to be too large and have limited accuracy in spotting dense, small objects in remote sensing scenarios. The reasons are: 1) Generative models treating continuous coordinate prediction as a token classification problem which fails to reflect the actual localization gap. 2) Current methods typically use CLIP pre-trained encoders that align text-image pairs at a global level, may not capture the fine-grained semantic object information necessary for accurate localization. To address these shortcomings, we introduce a lightweight generative model named LM-RSE (Localization Model with Refined Spotting Enhancement). Refined Spotting Enhancement includes the refinement of bounding box outputs and the integration of fine-grained semantic features. Specifically, we design a Bounding Box Refinement (BBR) approach that includes a special token, a Bounding Box decoder, and a custom regression loss function to refine the spotting precision and optimize the training process. Additionally, we propose a Fine-Grained Semantic Integration (FGSI) strategy, integrates a fine-grained image encoder, a Vision-Language Semantic Processing (VLSP) layer and a two-stage, full-parameter training strategy, all working together to effectively enhance the granularity of spotting. Building on this, we further refine the language-guided object localization task into two types: expression-guided and class-guided. For the former, we utilize the RSVGD dataset for evaluation and achieved state-of-the-art performance; for the latter, our evaluation results surpassed those of GeoChat. Our code and checkpoints will be released at https://github.com/Zhang-Peirong/LM-RSE.

Semantic segmentation of high-resolution remote sensing images is vital in downstream applications such as land-cover mapping, urban planning, and disaster assessment. Existing Transformer-based methods suffer from the constraint between accuracy and efficiency, while the recently proposed Mamba is renowned for being efficient. Therefore, to overcome the dilemma, we propose UNetMamba, a UNet-like semantic segmentation model based on Mamba. It incorporates a Mamba segmentation decoder (MSD) that can efficiently decode the complex information within high-resolution images, and a local supervision module (LSM), which is train-only but can significantly enhance the perception of local contents. Extensive experiments demonstrate that UNetMamba outperforms the state-of-the-art (SOTA) methods with mIoU increased by 0.87% on LoveDA and 0.39% on ISPRS Vaihingen while achieving high efficiency through the lightweight design, less memory footprint, and reduced computational cost. The source code is available at https://github.com/EnzeZhu2001/UNetMamba .