Self-supervised Video Hashing via Bidirectional Transformers
Authors:
- 1 June 2021
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 13544-13553
Abstract
Most existing unsupervised video hashing methods are built on unidirectional models with less reliable training objectives, which underuse the correlations among frames and the similarity structure between videos. To enable efficient scalable video retrieval, we propose a self-supervised video Hashing method based on Bidirectional Transformers (BTH). Based on the encoder-decoder structure of transformers, we design a visual cloze task to fully exploit the bidirectional correlations between frames. To unveil the similarity structure between unlabeled video data, we further develop a similarity reconstruction task by establishing reliable and effective similarity connections in the video space. Furthermore, we develop a cluster assignment task to exploit the structural statistics of the whole dataset such that more discriminative binary codes can be learned. Extensive experiments implemented on three public benchmark datasets, FCVID, ActivityNet and YFCC, demonstrate the superiority of our proposed approach.Keywords
Funding Information
- National Key Research and Development Program of China
- National Natural Science Foundation of China
This publication has 31 references indexed in Scilit:
- Query-Adaptive Hash Code Ranking for Large-Scale Multi-View Visual SearchIEEE Transactions on Image Processing, 2016
- Scalable Video Event Retrieval by Visual State Binary EmbeddingIEEE Transactions on Multimedia, 2016
- Deep hashing for compact binary codes learning2015
- Asymmetric Cyclical Hashing for Large Scale Image RetrievalIEEE Transactions on Multimedia, 2015
- ImageNet Large Scale Visual Recognition ChallengeInternational Journal of Computer Vision, 2015
- Weighted Component Hashing of Binary Aggregated Descriptors for Fast Visual SearchIEEE Transactions on Multimedia, 2015
- Multimodal Similarity-Preserving HashingIEEE Transactions on Pattern Analysis and Machine Intelligence, 2014
- Iterative Quantization: A Procrustean Approach to Learning Binary Codes for Large-Scale Image RetrievalIEEE Transactions on Pattern Analysis and Machine Intelligence, 2012
- Semi-Supervised Hashing for Large-Scale SearchIEEE Transactions on Pattern Analysis and Machine Intelligence, 2012
- Multiple feature hashing for real-time large scale near-duplicate video retrieval2011