Similarity Transitivity Broken-Aware Multi-Modal Hashing
- 5 June 2024
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Knowledge and Data Engineering
- Vol. 36 (11) , 7003-7014
Abstract
Due to the low storage cost and fast retrieval speed, multi-modal hashing, which maps the instances with different modal data-views into hash codes, has earned increasing research attention. Most existing supervised multi-modal hashing methods exploit the label information to define the similarities between instance pairs to supervise their training process. However, such methods ignore that the transitivity of their defined similarity has been broken in the multi-label scenarios, i.e., the instance $x$ is similar to the instance $y$ , and the instance $z$ is also similar to the instance $y$ , but $x$ may be not similar to $z$ , which will lead to fluctuations in the model optimization process and damage their retrieval performance. For example, in the first batch with instances $x$ and $y$ but without $z$ , the model will be optimized to make the hash codes of $x$ and $y$ similar to each other; In the second batch with instances $z$ and $y$ but without $x$ , the model will be optimized to make the hash codes of $z$ and $y$ similar to each other; In the third batch with the instances $x$ and $z$ but without $y$ , the model will be optimized to make the hash codes of $z$ and $x$ dissimilar to each other, meanwhile in this process, the hash codes of $z$ and $x$ may be dissimilar to that of $y$ which damage the optimizing results of the first two batches. Therefore, we propose a novel Similarity Transitivity Broken-aware Multi-modal Hashing, called STBMH, to solve this problem by adding a novel regularization loss into the original pair-wise loss. For each instance $x$ in a training batch, the regularization loss will take all instances in the training set into account. Extensive experiments on four widely used datasets show that the proposed method achieves better performance than the state-of-the-art baselines on multi-modal retrieval task.
Keywords
Funding Information
- National Natural Science Foundation of China (62172039, U21B2009, 62276110)
- Ministry of Industry and Information Technology of the People's Republic of China (CEIEC-2022-ZM02-0247)
- CCF-AFSG Research Fund (RF20210005)
- HUST and Pingan Property & Casualty Research
This publication has 52 references indexed in Scilit:
- SCH: Symmetric Consistent Hashing for cross-modal retrievalSignal Processing, 2023
- Cross-modal hashing with missing labelsNeural Networks, 2023
- Discrete online cross-modal hashingPattern Recognition, 2022
- Self-constraining and attention-based hashing network for bit-scalable cross-modal retrievalNeurocomputing, 2020
- Flexible Discrete Multi-view Hashing with Collective Latent Feature LearningNeural Processing Letters, 2020
- ImageNet Large Scale Visual Recognition ChallengeInternational Journal of Computer Vision, 2015
- Microsoft COCO: Common Objects in Context2014
- Multi-view anchor graph hashing2013
- Deep Learning to Hash with Multiple Representations2012
- The segmented and annotated IAPR TC-12 benchmarkComputer Vision and Image Understanding, 2010