Binary Set Embedding for Cross-Modal Retrieval
Authors:
- 27 September 2016
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Neural Networks and Learning Systems
- Vol. 28 (12) , 2899-2910
Abstract
Cross-modal retrieval is such a challenging topic that traditional global representations would fail to bridge the semantic gap between images and texts to a satisfactory level. Using local features from images and words from documents directly can be more robust for the scenario with large intraclass variations and small interclass discrepancies. In this paper, we propose a novel unsupervised binary coding algorithm called binary set embedding (BSE) to obtain meaningful hash codes for local features from the image domain and words from text domain. Understanding image features with the word vectors learned from the human language instead of the provided documents from data sets, BSE can map samples into a common Hamming space effectively and efficiently where each sample is represented by the sets of local feature descriptors from image and text domains. In particular, BSE explores relationship among local features in both feature level and image (text) level, which can balance the sensitivity of each other. Furthermore, a recursive orthogonalization procedure is applied to reduce the redundancy of codes. Extensive experiments demonstrate the superior performance of BSE compared with state-of-the-art cross-modal hashing methods using either image or text queries.Keywords
Funding Information
- Northumbria University
- National Natural Science Foundation of China (61528106)
This publication has 29 references indexed in Scilit:
- Evolutionary compact embedding for large-scale image classificationInformation Sciences, 2015
- Multiview Alignment Hashing for Efficient Image SearchIEEE Transactions on Image Processing, 2015
- Partially Shared Latent Factor Learning With Multiview DataIEEE Transactions on Neural Networks and Learning Systems, 2014
- On the Role of Correlation and Abstraction in Cross-Modal Multimedia RetrievalIEEE Transactions on Pattern Analysis and Machine Intelligence, 2013
- Multiview Vector-Valued Manifold Regularization for Multilabel Image ClassificationIEEE Transactions on Neural Networks and Learning Systems, 2013
- Semantic hashingInternational Journal of Approximate Reasoning, 2009
- Image retrievalACM Computing Surveys, 2008
- Hamming Embedding and Weak Geometric Consistency for Large Scale Image Search2008
- Distinctive Image Features from Scale-Invariant KeypointsInternational Journal of Computer Vision, 2004
- An optimal transformation for discriminant and principal component analysisIEEE Transactions on Pattern Analysis and Machine Intelligence, 1988