OmniTracker: Unifying Visual Object Tracking by Tracking-With-Detection
- 15 January 2025
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Pattern Analysis and Machine Intelligence
- Vol. 47 (4) , 3159-3174
Abstract
Visual Object Tracking (VOT) aims to estimate the positions of target objects in a video sequence, which is an important vision task with various real-world applications. Depending on whether the initial states of target objects are specified by provided annotations in the first frame or the categories, VOT could be classified as instance tracking (e.g., SOT and VOS) and category tracking (e.g., MOT, MOTS, and VIS) tasks. Different definitions have led to divergent solutions for these two types of tasks, resulting in redundant training expenses and parameter overhead. In this paper, combing the advantages of the best practices developed in both communities, we propose a novel tracking-with-detection paradigm, where tracking supplements appearance priors for detection and detection provides tracking with candidate bounding boxes for the association. Equipped with such a design, a unified tracking model, OmniTracker, is further presented to resolve all the tracking tasks with a fully shared network architecture, model weights, and inference pipeline, eliminating the need for task-specific architectures and reducing redundancy in model parameters. We conduct extensive experimentation on seven prominent tracking datasets of different tracking tasks, including LaSOT, TrackingNet, DAVIS16-17, MOT17, MOTS20, and YTVIS19, and demonstrate that OmniTracker achieves on-par or even better results than both task-specific and unified tracking models.Keywords
All Related Versions
Funding Information
- National Natural Science Foundation of China (62032006)
This publication has 119 references indexed in Scilit:
- SipMask: Spatial Information Preservation for Fast Image and Video Instance Segmentation2020
- Ocean: Object-Aware Anchor-Free Tracking2020
- Know Your Surroundings: Exploiting Scene Information for Object Tracking2020
- Conditional Convolutions for Instance Segmentation2020
- Learning What to Learn for Video Object Segmentation2020
- End-to-End Object Detection with Transformers2020
- Collaborative Video Object Segmentation by Foreground-Background Integration2020
- VideoMatch: Matching Based Video Object Segmentation2018
- TrackingNet: A Large-Scale Dataset and Benchmark for Object Tracking in the Wild2018
- The Hungarian method for the assignment problemNaval Research Logistics Quarterly, 1955