A Two-Stage Foveal Vision Tracker Based on Transformer Model
- 18 March 2024
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Cognitive and Developmental Systems
- Vol. 16 (4) , 1575-1588
Abstract
With the development of Transformer visual models, attention-based trackers have shown highly competitive performance in the field of object tracking. However, in some tracking scenarios, especially those with multiple similar objects, the performance of existing trackers is often not satisfactory. In order to improve the performance of trackers in such scenarios, inspired by the fovea vision structure and its visual characteristics, this paper proposes a novel Foveal Vision Tracker (FVT). FVT combines the process of human eye fixation and object tracking, pruning based on the distance to the object rather than attention scores. This pruning method allows the receptive field of the feature extraction network to focus on the object, excluding background interference. FVT divides the feature extraction network into two stages: local and global, and introduces the Local Recursive Module (LRM) and the View Elimination Module (VEM). LRM is used to enhance foreground features in the local stage, while VEM generates circular fovea-like visual field masks in the global stage and prunes tokens outside the mask, guiding the model to focus attention on high-information regions of the object. Experimental results on multiple object tracking datasets demonstrate that the proposed FVT achieves stronger object discrimination capability in the feature extraction stage, improves tracking accuracy and robustness in complex scenes, and achieves a significant accuracy improvement with an AO of 72.6% on the GOT-10k dataset.Keywords
Funding Information
- Natural Science Foundation of China (61871445, 61302156)
- Key R&D Foundation Project of Jiangsu Province (BE2016001-4)
This publication has 37 references indexed in Scilit:
- Sliced Recursive Transformer2022
- Joint Feature Learning and Relation Modeling for Tracking: A One-Stream Framework2022
- AiATrack: Attention in Attention for Transformer Visual Tracking2022
- Unsupervised foveal vision neural architecture with top-down attentionNeural Networks, 2021
- Ocean: Object-Aware Anchor-Free Tracking2020
- CornerNet: Detecting Objects as Paired Keypoints2018
- TrackingNet: A Large-Scale Dataset and Benchmark for Object Tracking in the Wild2018
- High Performance Visual Tracking with Siamese Region Proposal Network2018
- A Benchmark and Simulator for UAV Tracking2016
- Microsoft COCO: Common Objects in Context2014