Sketch-Based Image Retrieval Through Hypothesis-Driven Object Boundary Selection With HLR Descriptor
Authors:
- 8 May 2015
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Multimedia
- Vol. 17 (7) , 1045-1057
Abstract
The appearance gap between sketches and photo- realistic images is a fundamental challenge in sketch-based image retrieval (SBIR) systems. The existence of noisy edges on photo- realistic images is a key factor in the enlargement of the appearance gap and significantly degrades retrieval performance . To bridge the gap, we propose a framework consisting of a new line segment -based descriptor named histogram of line relationship (HLR) and a new noise impact reduction algorithm known as object boundary selection . HLR treats sketches and extracted edges of photo- realistic images as a series of piece-wise line segments and captures the relationship between them. Based on the HLR, the object boundary selection algorithm aims to reduce the impact of noisy edges by selecting the shaping edges that best correspond to the object boundaries. Multiple hypotheses are generated for descriptors by hypothetical edge selection. The selection algorithm is formulated to find the best combination of hypotheses to maximize the retrieval score; a fast method is also proposed. To reduce the distraction of false matches in the scoring process, two constraints on spatial and coherent aspects are introduced . We tested the HLR descriptor and the proposed framework on public datasets and a new image dataset of three million images, which we recently collected for SBIR evaluation purposes. We compared the proposed HLR with state-of-the-art descriptors (SHoG, GF-HOG). The experimental results show that our HLR descriptor outperforms them. Combined with the object boundary selection algorithm, our framework significantly improves SBIR performance.Keywords
Funding Information
- NSFC (61273274, 61370127)
- 973 Program
- National Key Technology RD Program of China (2012BAH01F03, NSFB4123104, FRFCU 2014JBZ004, Z131110001913143)
- Tsinghua-Tencent Joint Lab for IIT
This publication has 37 references indexed in Scilit:
- Similarity-Invariant Sketch-Based Image Retrieval in Large Databases2014
- Indexing billions of images for sketch-based retrieval2013
- Robust Spatial Consistency Graph Model for Partial Duplicate Image RetrievalIEEE Transactions on Multimedia, 2013
- Efficient Feature Detection and Effective Post-Verification for Large Scale Near-Duplicate Image SearchIEEE Transactions on Multimedia, 2011
- A bag-of-regions approach to sketch-based image retrieval2011
- Weakly Supervised Object Recognition and Localization with Invariant High Order Features2010
- Object retrieval with large vocabularies and fast spatial matching2007
- Inverted files for text search enginesACM Computing Surveys, 2006
- Understanding inverse document frequency: on theoretical arguments for IDFJournal of Documentation, 2004
- A Computational Approach to Edge DetectionIEEE Transactions on Pattern Analysis and Machine Intelligence, 1986