A Multi-Level Self-Distillation-Based Unified Tracker for Efficient RGB-T Tracking
- 27 February 2026
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Image Processing
- Vol. 35 (10577149) , 2407-2422
Abstract
RGB-Thermal (RGB-T) tracking enhances visual tracking robustness by combining RGB and thermal infrared (TIR) modalities, addressing limitations of RGB-only trackers under challenging conditions such as low light and appearance variations. However, most existing RGB-T trackers rely on complex fusion modules or modality-specific architectures, sacrificing efficiency for performance. In this paper, we propose a novel Multi-level Self-Distillation (MSD) framework that adapts a one-stream RGB tracker to the RGB-T setting without modifying the network architecture or adding any extra parameters. RGB and TIR inputs are jointly processed through a shared backbone, and training is guided by a combination of self-supervised and supervised objectives to enhance cross-modal feature representation. The self-supervised component includes a contrastive loss that aligns semantically consistent regions across template-search pairs, as well as a modality-gap alignment loss that reduces discrepancies between RGB and TIR features. These internal signals complement task-driven supervision, including an intermediate focal loss that strengthens early localization by enhancing shallow and mid-level features, modality-specific losses that preserve distinctive cues under partial modality degradation, and a fused tracking loss that drives final bounding box prediction. Comprehensive evaluations on LasHeR, RGBT234, and GTOT benchmarks demonstrate that MSD achieves state-of-the-art tracking accuracy while maintaining the computational efficiency of the original RGB tracker. Our work establishes a new paradigm in multi-modal tracking by demonstrating that optimized training strategies can outperform complex architectural modifications, offering significant practical advantages for real-world deployment.Keywords
Funding Information
- Natural Sciences and Engineering Research Council of Canada and in part by the Regroupement Stratégique en Microélectronique du Québec
This publication has 49 references indexed in Scilit:
- Two-stage Unidirectional Fusion Network for RGBT trackingKnowledge-Based Systems, 2025
- Modality-missing RGBT Tracking: Invertible Prompt Learning and High-quality BenchmarksInternational Journal of Computer Vision, 2024
- RGBT tracking: A comprehensive reviewInformation Fusion, 2024
- Learning a multimodal feature transformer for RGBT trackingSignal, Image and Video Processing, 2024
- Dual Knowledge Distillation for neural machine translationComputer Speech & Language, 2023
- Joint Feature Learning and Relation Modeling for Tracking: A One-Stream Framework2022
- Dual Siamese network for RGBT tracking via fusing predicted position mapsThe Visual Computer, 2021
- Object fusion tracking based on visible and infrared images: A comprehensive reviewInformation Fusion, 2020
- RGB-T object tracking: Benchmark and baselinePattern Recognition, 2019
- CornerNet: Detecting Objects as Paired Keypoints2018