A Self-Supervised Descriptor for Image Copy Detection
Authors:
- Ed PizziEPEd Pizzi
- Sreya Dutta RoySRSreya Dutta Roy
- Sugosh Nagavara RavindraSRSugosh Nagavara Ravindra
- Priya GoyalPGPriya Goyal
- Matthijs DouzeMDMatthijs Douze
Preprint
Abstract
Image copy detection is an important task for content moderation. We introduce SSCD, a model that builds on a recent self-supervised contrastive training objective. We adapt this method to the copy detection task by changing the architecture and training objective, including a pooling operator from the instance matching literature, and adapting contrastive learning to augmentations that combine images. Our approach relies on an entropy regularization term, promoting consistent separation between descriptor vectors, and we demonstrate that this significantly improves copy detection accuracy. Our method produces a compact descriptor vector, suitable for real-world web scale applications. Statistical information from a background image distribution can be incorporated into the descriptor. On the recent DISC2021 benchmark, SSCD is shown to outperform both baseline copy detection models and self-supervised architectures designed for image classification by huge margins, in all settings. For example, SSCD out-performs SimCLR descriptors by 48% absolute. Code is available at https://github.com/facebookresearch/sscd-copy-detection.Keywords
All Related Versions
This publication has 15 references indexed in Scilit:
- Distortion Agnostic Deep Watermarking2020
- CutMix: Regularization Strategy to Train Strong Classifiers With Localizable Features2019
- Billion-Scale Similarity Search with GPUsIEEE Transactions on Big Data, 2019
- HiDDeN: Hiding Data With Deep Networks2018
- CNN Image Retrieval Learns from BoW: Unsupervised Fine-Tuning with Hard Examples2016
- Image Search with Selective Match Kernels: Aggregation Across Single and Multiple ImagesInternational Journal of Computer Vision, 2015
- Large-Scale Image Phylogeny: Tracing Image Ancestral RelationshipsIEEE MultiMedia, 2013
- Hamming Embedding and Weak Geometric Consistency for Large Scale Image Search2008
- Total Recall: Automatic Query Expansion with a Generative Feature Model for Object Retrieval2007
- Video Google: a text retrieval approach to object matching in videos2003