The VISIONE Video Search System: Exploiting Off-the-Shelf Text Search Engines for Large-Scale Video Retrieval
Preprint
Abstract
In this paper, we describe in details VISIONE, a video search system that allows users to search for videos using textual keywords, occurrence of objects and their spatial relationships, occurrence of colors and their spatial relationships, and image similarity. These modalities can be combined together to express complex queries and satisfy user needs. The peculiarity of our approach is that we encode all the information extracted from the keyframes, such as visual deep features, tags, color and object locations, using a convenient textual encoding indexed in a single text retrieval engine. This offers great flexibility when results corresponding to various parts of the query (visual, text and locations) have to be merged. In addition, we report an extensive analysis of the system retrieval performance, using the query logs generated during the Video Browser Showdown (VBS) 2019 competition. This allowed us to fine-tune the system by choosing the optimal parameters and strategies among the ones that we tested.Keywords
All Related Versions
This publication has 11 references indexed in Scilit:
- Interactive Video Retrieval in the Age of Deep Learning – Detailed Evaluation of VBS 2019IEEE Transactions on Multimedia, 2020
- Interactive Search or Sequential Browsing? A Detailed Analysis of the Video Browser Showdown 2018ACM Transactions on Multimedia Computing, Communications, and Applications, 2019
- Object detection and classification: a joint selection and fusion strategy of deep convolutional neural network and SIFT point featuresMultimedia Tools and Applications, 2018
- On Influential Trends in Interactive Video Retrieval: Video Browser Showdown 2015–2017IEEE Transactions on Multimedia, 2018
- End-to-End Learning of Deep Visual Representations for Image RetrievalInternational Journal of Computer Vision, 2017
- Local and global approaches for unsupervised image annotationMultimedia Tools and Applications, 2016
- Video Stream Analysis in Clouds: An Object Detection and Classification Framework for High Performance Video AnalyticsIEEE Transactions on Cloud Computing, 2016
- Jointly Modeling Deep Video and Compositional Text to Bridge Vision and Language in a Unified FrameworkProceedings of the AAAI Conference on Artificial Intelligence, 2015
- Supervised Learning of Semantic Classes for Image Annotation and RetrievalIEEE Transactions on Pattern Analysis and Machine Intelligence, 2007
- CBSA: content-based soft annotation for multimodal image retrieval using bayes point machinesIEEE Transactions on Circuits and Systems for Video Technology, 2003