Quantifying and Narrowing the Unknown: Interactive Text-to-Video Retrieval via Uncertainty Minimization
Preprint
Abstract
Despite recent advances, Text-to-video retrieval (TVR) is still hindered by multiple inherent uncertainties, such as ambiguous textual queries, indistinct text-video mappings, and low-quality video frames. Although interactive systems have emerged to address these challenges by refining user intent through clarifying questions, current methods typically rely on heuristic or ad-hoc strategies without explicitly quantifying these uncertainties, limiting their effectiveness. Motivated by this gap, we propose UMIVR, an Uncertainty-Minimizing Interactive Text-to-Video Retrieval framework that explicitly quantifies three critical uncertainties-text ambiguity, mapping uncertainty, and frame uncertainty-via principled, training-free metrics: semantic entropy-based Text Ambiguity Score (TAS), Jensen-Shannon divergence-based Mapping Uncertainty Score (MUS), and a Temporal Quality-based Frame Sampler (TQFS). By adaptively generating targeted clarifying questions guided by these uncertainty measures, UMIVR iteratively refines user queries, significantly reducing retrieval ambiguity. Extensive experiments on multiple benchmarks validate UMIVR's effectiveness, achieving notable gains in Recall@1 (69.2\% after 10 interactive rounds) on the MSR-VTT-1k dataset, thereby establishing an uncertainty-minimizing foundation for interactive TVR.Keywords
All Related Versions
This publication has 8 references indexed in Scilit:
- Are Synthetic Videos Useful? A Benchmark for Retrieval-Centric Evaluation of Synthetic Videos2025
- Continual Text-to-Video Retrieval with Frame Fusion and Task-Aware Routing2025
- Detecting hallucinations in large language models using semantic entropyNature, 2024
- Prototype-based Aleatoric Uncertainty Quantification for Cross-modal Retrieval2023
- Tencent Text-Video Retrieval: Hierarchical Cross-Modal Interactions With Multi-Level RepresentationsIEEE Access, 2022
- CLIP4Clip: An empirical study of CLIP for end to end video clip retrieval and captioningNeurocomputing, 2022
- Aleatoric and epistemic uncertainty in machine learning: an introduction to concepts and methodsMachine Learning, 2021
- Relevance feedback: a power tool for interactive content-based image retrievalIEEE Transactions on Circuits and Systems for Video Technology, 1998