ACNet: Approaching-and-Centralizing Network for Zero-Shot Sketch-Based Image Retrieval
Preprint
Abstract
The huge domain gap between sketches and photos and the highly abstract sketch representations pose challenges for sketch-based image retrieval (\underline{SBIR}). The zero-shot sketch-based image retrieval (\underline{ZS-SBIR}) is more generic and practical but poses an even greater challenge because of the additional knowledge gap between the seen and unseen categories. To simultaneously mitigate both gaps, we propose an \textbf{A}pproaching-and-\textbf{C}entralizing \textbf{Net}work (termed "\textbf{ACNet}") to jointly optimize sketch-to-photo synthesis and the image retrieval. The retrieval module guides the synthesis module to generate large amounts of diverse photo-like images which gradually approach the photo domain, and thus better serve the retrieval module than ever to learn domain-agnostic representations and category-agnostic common knowledge for generalizing to unseen categories. These diverse images generated with retrieval guidance can effectively alleviate the overfitting problem troubling concrete category-specific training samples with high gradients. We also discover the use of proxy-based NormSoftmax loss is effective in the zero-shot setting because its centralizing effect can stabilize our joint training and promote the generalization ability to unseen categories. Our approach is simple yet effective, which achieves state-of-the-art performance on two widely used ZS-SBIR datasets and surpasses previous methods by a large margin.Keywords
All Related Versions
This publication has 9 references indexed in Scilit:
- Zero-shot sketch-based image retrieval with structure-aware asymmetric disentanglementComputer Vision and Image Understanding, 2022
- Learning semantic priors for texture-realistic sketch-to-image synthesisNeurocomputing, 2021
- CrossATNet - a novel cross-attention based framework for sketch-based image retrievalImage and Vision Computing, 2020
- Progressive Cross-Modal Semantic Network for Zero-Shot Sketch-Based Image RetrievalIEEE Transactions on Image Processing, 2020
- Enhancing Sketch-Based Image Retrieval by CNN Semantic Re-rankingIEEE Transactions on Cybernetics, 2019
- A performance evaluation of gradient field HOG descriptor for sketch based image retrievalComputer Vision and Image Understanding, 2013
- Photosketcher: Interactive Sketch-Based Image SynthesisIEEE Computer Graphics and Applications, 2011
- Sketch-Based Image Retrieval: Benchmark and Bag-of-Features DescriptorsIEEE Transactions on Visualization and Computer Graphics, 2010
- Fast image blending using watersheds and graph cutsImage and Vision Computing, 2009