A Visual-Linguistic Approach for Robust RGB-Thermal Tracking with Dynamic Template Adaptation
- 27 February 2026
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Multimedia
- Vol. PP (15209210) , 1-11
Abstract
RGB-T tracking seeks to improve tracking robust ness in complex environments by exploiting the complementary advantages of RGB and thermal infrared (TIR) modalities. Nevertheless, current RGB-T tracking methods encounter two critical limitations stemming from their reliance on fixed template images for search region matching. First, the resemblance between the initial fixed template and the target in later frames diminishes as time goes on, causing tracking reliability to decline. Second, background clutter within template bounding boxes introduces disruptive noise that compromises tracking accuracy. To address these challenges, we propose a new referring RGB-T tracking approach that integrates visual-linguistic cues to enhance multimodal data, enabling background noise elimination in the template and dynamic template image updating through adaptive mechanisms. Additionally, to facilitate research in language guided multimodal tracking, we construct a large-scale dataset named Refer-RGBT, which comprises 1508 multimodal video pairs with synchronized RGB/TIR sequences of diverse objects. Each sequence is annotated by descriptive textual captions that detail their appearance and actions. Extensive evaluations on the Refer-RGBT dataset demonstrate that our proposed approach achieves cutting-edge performance compared to the state-of-the art (SOTA) methods.Keywords
Funding Information
- National Natural Science Foundation of China (62371310)
- Basic and Applied Basic Research Foundation of Guangdong Province (2023A1515011236)
- Shenzhen Science and Technology Program (JCYJ20241202124415021)
- Stable Support Project of Shenzhen (20231122122722001)
- Scientific Foundation for Youth Scholars of Shenzhen University
This publication has 43 references indexed in Scilit:
- LLDiffusion: Learning degradation representations in diffusion models for low-light image enhancementPattern Recognition, 2025
- Joint Feature Learning and Relation Modeling for Tracking: A One-Stream Framework2022
- Backbone is All Your Need: A Simplified Architecture for Visual Object Tracking2022
- Learning Adaptive Attribute-Driven Representation for Real-Time RGB-T TrackingInternational Journal of Computer Vision, 2021
- Challenge-Aware RGBT Tracking2020
- RGB-T object tracking: Benchmark and baselinePattern Recognition, 2019
- LaSOT: A High-Quality Benchmark for Large-Scale Single Object Tracking2019
- Generalized Intersection Over Union: A Metric and a Loss for Bounding Box Regression2019
- CornerNet: Detecting Objects as Paired Keypoints2018
- Tracking by Natural Language Specification2017