Domain Adaptation-Aware Transformer for Hyperspectral Object Tracking
- 4 April 2024
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Circuits and Systems for Video Technology
- Vol. 34 (9) , 8041-8052
Abstract
Visual object tracking in natural scenes is a popular but challenging task, owing to the difficulties of feature representation from various changes of the targets, such as size change, deformation, illumination change, rotations, motion blur, background clutter, etc. High-speed hyperspectral imaging systems capture hyperspectral videos (HSVs) in wide spectral ranges and provide abundant spectral and spatial information to tell targets apart from backgrounds, alleviating the model drift in appearance-based tracking methods. However, different hyperspectral imagers, such as near-infrared (NIR), red-to-near-infrared (RedNIR), and visible (VIS), obtain heterogeneous types of data that could not be handled by common object trackers. In this paper, a domain adaptive Transformer framework is proposed for hyperspectral object tracking. Considering the HSVs are from different types of sensors, their heterogeneous features are learned in an adversarial way by domain label reverse learning with a gradient reversed layer. To fully utilize the spectral information in HSV frames, a band-wise spatial attention module (BSAM) is designed to emphasize the salient area near the target of interest. We adopt a Siamese-like Transformer tracker as the main structure for tracking. Our tracker outperforms top-ranking methods on a hyperspectral object tracking benchmark dataset containing three types, 87 hyperspectral videos in total. The comparison experiments validate the effectiveness of the proposed method. The source code and trained models of this work will be publicly available soon at https://github.com/LianYi233/Trans-DAT.Keywords
Funding Information
- Key Scientific Technological Innovation Research Project by the Ministry of Education;
- Joint Funds of the National Natural Science Foundation of China (U22B2054)
This publication has 53 references indexed in Scilit:
- Self-Supervised Image Denoising for Real-World Images With Context-Aware TransformerIEEE Access, 2023
- Squeeze-and-Excitation Networks2018
- SCA-CNN: Spatial and Channel-Wise Attention in Convolutional Networks for Image Captioning2017
- Fully-Convolutional Siamese Networks for Object Tracking2016
- Beyond Correlation Filters: Learning Continuous Convolution Operators for Visual Tracking2016
- Deep Residual Learning for Image Recognition2016
- A Scale Adaptive Kernel Correlation Filter Tracker with Feature Integration2015
- Adaptive Color Attributes for Real-Time Visual Tracking2014
- An adaptive color-based particle filterImage and Vision Computing, 2003
- Real-time tracking of non-rigid objects using mean shift2002