Deep Neighborhood-Preserving Hashing With Quadratic Spherical Mutual Information for Cross-Modal Retrieval
Top Cited Papers
- 1 January 2024
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Multimedia
- Vol. 26 (15209210) , 6361-6374
Abstract
Driven by the high nonlinearity of deep neural networks, deep hashing has achieved the pictured great potential in cross-modal retrieval applications, significantly bridging the modality gap. Current deep cross-modal hashing usually utilizes affinity matching or local ranking to capture the local semantic relationships in the learned common space, leading to high neighborhood ambiguity. Simultaneously, most of these frameworks utilize additional regularization terms or margin thresholds to enhance the overall performance, in which searching the model's hyper-parameters under mass training data would have a substantial overhead. In this paper, with a novel extension of information-theoretic measures, a novel deep cross-modal hashing method, named Deep Neighborhood-preserving Hashing (DNpH), is designed to learn a highly separable discrete space, effectively mitigating the semantic gap across different modalities. Specifically, to minimize neighborhood ambiguity, the Quadratic Spherical Mutual Information (QSMI) is first introduced into deep cross-modal hashing to separate neighbors and non-neighbors well, while it is free of tuning parameters during model training compared with other similarity measures. To optimize quadratic mutual information loss smoothly, a square clamping method is developed to improve the stability of model optimization, avoiding converging on bad local optimum. Besides, two transformer encoders are exploited as feature extractors for multi-modal samples to learn the informative semantic representations. Finally, we compare our proposed DNpH framework with various state-of-the-art cross-modal hashing on four public datasets, and large amounts of experiment results demonstrate our contributions and show that DNpH outperforms the compared baselines on different evaluation metrics. The corresponding code is available at https://github.com/QinLab-WFU/DNpH.Keywords
Funding Information
- National Natural Science Foundation of China (62302338, 62006174)
- Natural Science Foundation of Shandong Province (ZR2022QF046, ZR2021MF026, ZR2023MF033)
- Natural Science Foundation of Chongqing Municipality (2023NSCQ-MSX1645)
- Science and Technology Research Program of Chongqing Municipal Education Commission (KJQN202200551)
This publication has 60 references indexed in Scilit:
- A survey on sentiment analysis methods, applications, and challengesArtificial Intelligence Review, 2022
- Multi-label enhancement based self-supervised deep cross-modal hashingNeurocomputing, 2022
- Deep supervised hashing using quadratic spherical mutual information for efficient image retrievalSignal Processing: Image Communication, 2021
- Improving the performance of lightweight CNNs for binary classification using quadratic mutual information regularizationPattern Recognition, 2020
- Self-constraining and attention-based hashing network for bit-scalable cross-modal retrievalNeurocomputing, 2020
- Grad-CAM: Visual Explanations from Deep Networks via Gradient-Based LocalizationInternational Journal of Computer Vision, 2019
- Learning Deep Representations with Probabilistic Knowledge Transfer2018
- Cross-Modal Hamming Hashing2018
- Deep Cross-Modal Hashing2017
- Deep Residual Learning for Image Recognition2016