Yunpeng Liu

1. Cross-modal image matching aims to establish spatial correspondences between heterogeneous images acquired from different sensors for accurate alignment at the pixel or region level (Y. Ye and S...

Existing deep Thermal InfraRed (TIR) trackers often use RGB datasets for training due to the lack of large-scale labeled TIR datasets. However, the performance of these methods on TIR image sequences is significantly degraded, because of the domain shift problem between the RGB and TIR datasets. To solve this problem, in this paper, we propose an unsupervised Dual-level Domain Adaptation TIR Tracking framework (DDAT), which can benefit from training on large-scale labeled RGB datasets and unlabeled TIR datasets. Specifically, to transfer the useful knowledge learned from RGB dataset to TIR tracking, we first propose an adversarial-based adaptation module on both the semantic-level and the feature-level. While the semantic-level adaptation can reduce the semantic gap between the TIR and RGB tracking tasks, the feature-level adaptation can learn domain-invariant features for more robust tracking. Second, we propose a partial domain adaptation module to alleviate the negative transfer problem because the RGB and TIR tracking domains have a non-identical class and feature spaces. Instead of aligning the entire feature space, this module adaptively selects partial similarity samples and features for alignment, thus getting more fine-grained aligned results. Third, we collect a currently largest-scale unlabeled TIR dataset to train the proposed framework. Extensive experiments on five TIR tracking benchmarks demonstrate the proposed method is effective and sets a new state-of-the-art.

Optical-DSM segmentation leverages auxiliary Digital Surface Model (DSM) data to enhance optical image segmentation, but faces challenges in effectively extracting unimodal features and multimodal fusion. To address this, we propose the Detail-enhancement Dual-domain Fusion Network (D $^{2}$ FNet). First, D $^{2}$ FNet employs a double backbone strategy to fully extract unimodal semantic features: a powerful DINOv2-s extracts rich semantic features from optical imagery, while a lightweight ResNet18 efficiently captures essential height-related information from DSM. Second, D $^{2}$ FNet introduces a cascaded multimodal fusion framework, comprising the Dual-domain Fusion Block (DFB) and the Detail Enhancement Fusion Block (DEFB). Within the DFB, we employ the proposed Feature Interaction Fusion Block (FIFB) to perform initial multimodal feature alignment and fusion in both the spatial and frequency domains, enhancing the model's understanding of the macroscopic object structures. Recognizing the importance of local details for dense segmentation tasks, DEFB employs novel Variable Receptive Field Convolution (VRFC) modules to enhance the detail representation of both the initial fusion features and the DSM features. These detail-enhanced features are fed into the FIFB module to achieve further multimodal alignment and fusion. Extensive experiments on the ISPRS Potsdam and Vaihingen datasets demonstrate that our method achieves mIoU of 86.99% and 84.29%, respectively. It not only outperforms comparative approaches in segmentation performance but also reduces model parameters by 79.6% and computational cost by 85.6%, thereby validating the effectiveness of the proposed network.