TimeMarker: A Versatile Video-LLM for Long and Short Video Understanding with Superior Temporal Localization Ability
Preprint
Abstract
Rapid development of large language models (LLMs) has significantly advanced multimodal large language models (LMMs), particularly in vision-language tasks. However, existing video-language models often overlook precise temporal localization and struggle with videos of varying lengths. We introduce TimeMarker, a versatile Video-LLM designed for high-quality dialogue based on video content, emphasizing temporal localization. TimeMarker integrates Temporal Separator Tokens to enhance temporal awareness, accurately marking specific moments within videos. It employs the AnyLength mechanism for dynamic frame sampling and adaptive token merging, enabling effective handling of both short and long videos. Additionally, TimeMarker utilizes diverse datasets, including further transformed temporal-related video QA datasets, to bolster its temporal understanding capabilities. Image and interleaved data are also employed to further enhance the model's semantic perception ability. Evaluations demonstrate that TimeMarker achieves state-of-the-art performance across multiple benchmarks, excelling in both short and long video categories. Our project page is at \url{https://github.com/TimeMarker-LLM/TimeMarker/}.Keywords
All Related Versions
This publication has 16 references indexed in Scilit:
- Kangaroo: A Powerful Video-Language Model Supporting Long-context Video InputInternational Journal of Computer Vision, 2026
- VideoChat: chat-centric video understandingScience China Information Sciences, 2025
- Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysis2025
- Video-XL: Extra-Long Vision Language Model for Hour-Scale Video Understanding2025
- ShareGPT4V: Improving Large Multi-modal Models with Better Captions2024
- Improved Baselines with Visual Instruction Tuning2024
- MVBench: A Comprehensive Multi-modal Video Understanding Benchmark2024
- CapsFusion: Rethinking Image-Text Data at Scale2024
- Unmasked Teacher: Towards Training-Efficient Video Foundation Models2023
- Deep Learning-Based Action Detection in Untrimmed Videos: A SurveyIEEE Transactions on Pattern Analysis and Machine Intelligence, 2022