Toward Real-Time UAV Tracking With Adaptive and Background-Aware Vision Transformers
- 12 January 2026
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Circuits and Systems for Video Technology
- Vol. 36 (5) , 7395-7410
Abstract
In Unmanned Aerial Vehicle (UAV) tracking, discriminative correlation filters (DCF) are popular for their speed, making them ideal for real-time use with limited resources. Recently, lightweight convolutional neural networks (CNNs) have offered a new approach. Through filter pruning, these CNNs maintain high accuracy and efficiency, making them a strong alternative, especially for greater precision. Despite these advancements, the potential of pure vision transformers (ViTs) in UAV tracking remains largely untapped, especially based on the paradigm of conditional computation. In this work, we introduce an adaptive and background-aware Vision Transformer (Aba-ViT) and leverage it to develop a real-time UAV tracking framework called Aba-ViTrack. The proposed Aba-ViT exploits an adaptive and background-aware token computation method to reduce inference time. This approach adaptively discards tokens based on learned halting probabilities, which a priori are higher for background tokens than target ones. To further improve efficiency, this paper proposes a novel classroom-style learning (CSL) approach, where robustness knowledge is transmitted vertically from teacher to students, and generalization capability is enhanced through horizontal mutual learning among students. This method is used to compress Aba-ViTrack, resulting in Aba-ViTrack++. The upgraded version achieves a better balance between accuracy and efficiency in real-time UAV tracking. This version achieves a better balance between accuracy and efficiency for real-time UAV tracking. Extensive experiments on six UAV tracking benchmarks demonstrate that the proposed method achieves state-of-the-art performance in UAV tracking. The code is available at https://github.com/xyyang317/Aba-ViTrack.Keywords
Funding Information
- National Science Foundation (2024GXNSFAA010484)
- National Natural Science Foundation of China (62466013)
- National Natural Science Foundation of China (62206123)
- National Natural Science Foundation of China (62066042)
- National Natural Science Foundation of China (62176170)
This publication has 73 references indexed in Scilit:
- How to Train the Teacher Model for Effective Knowledge Distillation2024
- Joint Feature Learning and Relation Modeling for Tracking: A One-Stream Framework2022
- FEAR: Fast, Efficient, Accurate and Robust Visual Tracker2022
- Ocean: Object-Aware Anchor-Free Tracking2020
- Know Your Surroundings: Exploiting Scene Information for Object Tracking2020
- End-to-End Object Detection with Transformers2020
- VisDrone-SOT2018: The Vision Meets Drone Single-Object Tracking Challenge Results2019
- The Unmanned Aerial Vehicle Benchmark: Object Detection and Tracking2018
- TrackingNet: A Large-Scale Dataset and Benchmark for Object Tracking in the Wild2018
- Microsoft COCO: Common Objects in Context2014