Mutual Enhancement Fusion Tracker for Video-Level RGB-T Object Tracking
Open Access
Abstract
RGB and thermal visual tracking is a highly effective solution for complex environments. However, existing multimodal trackers still face significant limitations. They often apply uniform fusion weights across the entire image. They also ignore the distinct physical properties of different modalities and lack historical temporal modeling. To address these issues, we propose a novel video-level framework named Mutual Enhancement Fusion Tracker (MEFT). First, we introduce a modality-aware mutual enhancement module. This module explicitly transfers visual texture from the RGB modality to the thermal modality. Simultaneously, it transfers heat saliency from the thermal modality to the RGB modality. Second, we design an adaptive spatial gating network. This network generates normalized pixel-level weights to ensure an optimal combination of features at different locations. Finally, the fused representations are processed by a spatiotemporal sequence backbone. Extensive experiments on four benchmark datasets demonstrate that our method achieves competitive performance with state-of-the-art RGB-T trackers. For instance, our tracker achieves 60.3% SR on LasHeR and 67.5% MSR on RGBT234. MEFT significantly improves both tracking accuracy and overall reliability in a wide range of challenging real-world scenarios.Keywords
Funding Information
- National Natural Science Foundation of China (62376224)
- Innovation Capability Support Program of Shaanxi (2026RS-CXTD-16)
This publication has 48 references indexed in Scilit:
- HATrack: Cross-modal fusion RGBT tracking with heterogeneous adapterExpert Systems with Applications, 2026
- Cross-modal guiding attention for RGBT trackingInformation Fusion, 2025
- Object Tracking: A Comprehensive Survey of Deep and Traditional MethodsArchives of Computational Methods in Engineering, 2025
- RAMR: A role-adaptive modality recalibration network for RGBT trackingExpert Systems with Applications, 2025
- Mining representative tokens via transformer-based multi-modal interaction for RGB-T trackingPattern Recognition, 2025
- Multi-modal adapter for RGB-T trackingInformation Fusion, 2025
- UniRTL: A universal RGBT and low-light benchmark for object trackingPattern Recognition, 2024
- Correlation filter based single object tracking: A reviewInformation Fusion, 2024
- Multi-modal visual tracking based on textual generationInformation Fusion, 2024
- RGB-T object tracking: Benchmark and baselinePattern Recognition, 2019