Data-Free Sketch-Based Image Retrieval
Authors:
- 1 June 2023
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 12084-12093
Abstract
Rising concerns about privacy and anonymity preservation of deep learning models have facilitated research in data-free learning (DFL). For the first time, we identify that for data-scarce tasks like Sketch-Based Image Retrieval (SBIR), where the difficulty in acquiring paired photos and hand-drawn sketches limits data-dependent cross-modal learning algorithms, DFL can prove to be a much more practical paradigm. We thus propose Data-Free (DF)-SBIR, where, unlike existing DFL problems, pre-trained, single-modality classification models have to be leveraged to learn a cross-modal metric-space for retrieval without access to any training data. The widespread availability of pre-trained classification models, along with the difficulty in acquiring paired photo-sketch datasets for SBIR justify the practicality of this setting. We present a methodology for DF-SBIR, which can leverage knowledge from models independently trained to perform classification on photos and sketches. We evaluate our model on the Sketchy, TU-Berlin, and QuickDraw benchmarks, designing a variety of baselines based on state-of-the-art DFL literature, and observe that our method surpasses all of them by significant margins. Our method also achieves mAPs competitive with data-dependent approaches, all the while requiring no training data. Implementation is available at https://github.com/abhrac/data-free-sbir.Keywords
All Related Versions
This publication has 27 references indexed in Scilit:
- No Fuss Distance Metric Learning Using Proxies2017
- Deep Sketch Hashing: Fast Free-Hand Sketch-Based Image Retrieval2017
- The sketchy databaseACM Transactions on Graphics, 2016
- Deep Residual Learning for Image Recognition2016
- Understanding deep image representations by inverting them2015
- Sketch based Image Retrieval using Learned KeyShapes (LKS)2015
- Sketch based image retrieval using a soft computation of the histogram of edge local orientations (S-HELO)2014
- How do humans sketch objects?ACM Transactions on Graphics, 2012
- Robust Optimization2009
- Learning a Similarity Metric Discriminatively, with Application to Face Verification2005