Deep Semantic-Alignment Hashing for Unsupervised Cross-Modal Retrieval
Authors:
Open Access
- 8 June 2020
- conference paper
- Published by Association for Computing Machinery (ACM)
- p. 44-52
Abstract
Deep hashing methods have achieved tremendous success in cross-modal retrieval, due to its low storage consumption and fast retrieval speed. In real cross-modal retrieval applications, it's hard to obtain label information. Recently, increasing attention has been paid to unsupervised cross-modal hashing. However, existing methods fail to exploit the intrinsic connections between images and their corresponding descriptions or tags (text modality). In this paper, we propose a novel Deep Semantic-Alignment Hashing (DSAH) for unsupervised cross-modal retrieval, which sufficiently utilizes the co-occurred image-text pairs. DSAH explores the similarity information of different modalities and we elaborately design a semantic-alignment loss function, which elegantly aligns the similarities between features with those between hash codes. Moreover, to further bridge the modality gap, we innovatively propose to reconstruct features of one modality with hash codes of the other one. Extensive experiments on three cross-modal retrieval datasets demonstrate that DSAH achieves the state-of-the-art performance.Keywords
Funding Information
- the national key Research and Development Program of China (NO.2018YFC0825102?No.2019YFC0850202)
- the Strategic Priority Research Program of the Chinese Academy of Sciences (XDC02050200)
- the NUDT Research Program (KJ1703)
This publication has 19 references indexed in Scilit:
- A Survey on Learning to HashIEEE Transactions on Pattern Analysis and Machine Intelligence, 2017
- Multimodal Discriminative Binary Embedding for Large-Scale Cross-Modal RetrievalIEEE Transactions on Image Processing, 2016
- Correlation Autoencoder Hashing for Supervised Cross-Modal Search2016
- Semantics-preserving hashing for cross-view retrieval2015
- ImageNet Large Scale Visual Recognition ChallengeInternational Journal of Computer Vision, 2015
- Latent semantic sparse hashing for cross-modal similarity search2014
- On the Role of Correlation and Abstraction in Cross-Modal Multimedia RetrievalIEEE Transactions on Pattern Analysis and Machine Intelligence, 2013
- Inter-media hashing for large-scale retrieval from heterogeneous data sources2013
- NUS-WIDE2009
- Object recognition from local scale-invariant features1999