SAMURAI: Motion-Aware Memory for Training-Free Visual Object Tracking With SAM 2
Top Cited Papers
Open Access
- 13 January 2026
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Image Processing
- Vol. 35 (10577149) , 970-982
Abstract
The Segment Anything Model 2 (SAM 2) has demonstrated exceptional performance in object segmentation tasks but encounters challenges in visual object tracking, particularly in handling crowded scenes with fast-moving or self-occluding objects. Additionally, its fixed-window memory mechanism indiscriminately retains past frames, leading to error accumulation. This issue results in incorrect memory retention during occlusions, causing the model to condition future predictions on unreliable features and leading to identity switches or drift in crowded scenes. This paper introduces SAMURAI, an enhanced adaptation of SAM 2 that integrates temporal motion cues with a novel motion-aware memory selection strategy. SAMURAI effectively predicts object motion and refines mask selection, achieving robust and precise tracking without requiring retraining or fine-tuning. It demonstrates strong training-free performance across multiple VOT benchmark datasets, underscoring its generalization capability. SAMURAI achieves state-of-the-art performance on LaSOText, GOT-10k, and TrackingNet, while also delivering competitive results on LaSOT, VOT2020-ST, VOT2022-ST, and VOS benchmarks such as SA-V. These results highlight SAMURAI’s robustness in complex tracking scenarios and its potential for real-world applications in dynamic environments with an optimized memory selection mechanism. Code and results are available at https://github.com/yangchris11/samurai.Keywords
This publication has 49 references indexed in Scilit:
- Tracking Meets LoRA: Faster Training, Larger Model, Stronger Performance2024
- The Tenth Visual Object Tracking VOT2022 Challenge Results2023
- Joint Feature Learning and Relation Modeling for Tracking: A One-Stream Framework2022
- ByteTrack: Multi-object Tracking by Associating Every Detection Box2022
- AiATrack: Attention in Attention for Transformer Visual Tracking2022
- XMem: Long-Term Video Object Segmentation with an Atkinson-Shiffrin Memory Model2022
- LaSOT: A High-quality Large-scale Single Object Tracking BenchmarkInternational Journal of Computer Vision, 2020
- The Eighth Visual Object Tracking VOT2020 Challenge Results2020
- TrackingNet: A Large-Scale Dataset and Benchmark for Object Tracking in the Wild2018
- Learning Dynamic Memory Networks for Object Tracking2018