PIC-CMH: Efficient Prompt-Infused Continual Cross-Modal Hashing
- 12 January 2026
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Multimedia
- Vol. 28 (15209210) , 2654-2667
Abstract
Cross-modal hashing models face significant challenges in handling continuous data growth, particularly in balancing the plasticity to learn new knowledge and the stability to retain prior cross-modal knowledge. Existing studies partially address this by maintaining previous mappings or extending hash codes, but struggle to reconcile plasticity and stability while requiring heavy parameter optimization. To tackle this, we propose an efficient Prompt-Infused Continual Cross-Modal Hashing (PIC-CMH) approach designed for hash learning with the continuous growth of multi-modal data and emerging knowledge. Specifically, PIC-CMH introduces a finite set of learnable multi-modal prompts, including global and task-specific expert prompts, which work in synergy with multi-modal representations. All prompts are optimized with the hash functions via backpropagation after Gaussian initialization. Global prompts stay learnable throughout, linking tasks, while expert prompts are updated only within their tasks, facilitating knowledge acquisition and mitigating catastrophic forgetting in continual learning. By freezing the pre-trained models used for multi-modal representations, continual learning is confined to the lightweight multi-modal prompts and hash functions, significantly reducing computational overhead. Extensive experiments demonstrate that PIC-CMH effectively addresses the stability-plasticity trade-off in cross-modal hash learning, delivering high retrieval accuracy with low computational cost and a simple yet efficient architecture. The source codes and datasets are available at https://github.com/styx29-0/PIC-CMH.Keywords
Funding Information
- National Natural Science Foundation of China (62172263)
- Fundamental Research Funds for the Central Universities
This publication has 52 references indexed in Scilit:
- Unsupervised cross-modal hashing retrieval via Dynamic Contrast and OptimizationEngineering Applications of Artificial Intelligence, 2024
- Structure-aware contrastive hashing for unsupervised cross-modal retrievalNeural Networks, 2024
- Deep continual hashing with gradient-aware memory for cross-modal retrievalPattern Recognition, 2022
- Online supervised collective matrix factorization hashing for cross-modal retrievalApplied Intelligence, 2022
- Online unsupervised cross-view discrete hashing for large-scale retrievalApplied Intelligence, 2022
- Pre-trained models: Past, present and futureAI Open, 2021
- GrowBit: Incremental Hashing for Cross-Modal Retrieval2019
- Memory Aware Synapses: Learning What (not) to Forget2018
- Riemannian Walk for Incremental Learning: Understanding Forgetting and Intransigence2018
- Microsoft COCO: Common Objects in Context2014