Fast and Robust Video Deduplication
Open Access
- 7 May 2023
- conference paper
- Published by Association for Computing Machinery (ACM)
- p. 160
Abstract
The popularity of social media networks and mobile devices have skyrocketed in recent years. This has led to the rapid increase in the video content being recorded and uploaded to online platforms like TikTok, YouTube, etc. As a result, a rise in the amount of illegal pirate videos can be observed. These pirate videos have the same content as original videos but with minor editing effects and variations in coding. The task of finding such duplicate videos is known as video deduplication. Also, storing this huge amount of video data is a challenging issue. Generally, video deduplication systems rely on feature extraction followed by computing a similarity score. The ideal goal is to develop a discriminative feature to differentiate among similar videos. In this work, we present a robust and lightweight video deduplication that can find duplicate videos among a large video repository extremely quickly within a few milliseconds. We propose a robust and highly discriminative feature that was generated using fisher vector and a thumbnail feature. The fisher vector was generated by applying fisher vector aggregation on Scale-Invariant Feature Transform (SIFT) keypoints with Gaussian Mixture Model (GMM) as the generative model. GMM was trained via SIFT key-points extracted from frames uniformly sampled from videos. For the thumbnail feature, each frame is resized to a 12x12 resolution. We reduce the dimensions of the fisher vector and thumbnail feature to 32-d by using principal component analysis (PCA), leaving us with our proposed deduplication features. A large repository of 1 million frames was generated by extracting frames from videos at uniform steps. Each frame in the repository was assigned a unique global timestamp. For fast video retrieval from the large repository, a multiple k-d tree setup was designed that trains a separate k-d tree for the fisher vector and the thumbnail feature. KNN search was used to search the k-d tree which returns the samples nearest to the query. In our case, we combine the nodes retrieved from both the k-d trees for better retrieval results. The k-d tree retrieval provides us with a frame retrieval system. For video retrieval, we develop a data pruning strategy that utilizes the sequence ID and timestamp information to accurately retrieve duplicate videos. CDVS dataset [1] was used for training the GMM. The 1 million frame repository and the test data were generated using the large-scale FIVR-200K dataset [2]. Experimental results show that our retrieval results are highly accurate and the system can process a query video within a few milliseconds.Keywords
Funding Information
- National Science Foundation (2148382)
This publication has 2 references indexed in Scilit:
- FIVR: Fine-Grained Incident Video RetrievalIEEE Transactions on Multimedia, 2019
- Overview of the MPEG-CDVS StandardIEEE Transactions on Image Processing, 2015