Learning Together Securely: Prototype-Based Federated Multi-Modal Hashing for Safe and Efficient Multi-Modal Retrieval
Authors:
Open Access
- 11 April 2025
- journal article
- Published by Association for the Advancement of Artificial Intelligence (AAAI) in Proceedings of the AAAI Conference on Artificial Intelligence
- Vol. 39 (21) , 23108-23116
Abstract
With the proliferation of multi-modal data, safe and efficient multi-modal hashing retrieval has become a pressing research challenge, particularly due to concerns over data privacy during centralized processing. To address this, we propose Prototype-based Federated Multi-modal Hashing (PFMH), an innovative framework that seamlessly integrates federated learning with multi-modal hashing techniques. PFMH achieves fine-grained fusion of heterogeneous multi-modal data, enhancing retrieval accuracy while ensuring data privacy through prototype-based communication, thereby reducing communication costs and mitigating risks of data leakage. Furthermore, using a prototype completion strategy, PFMH tackles class imbalance and statistical heterogeneity in multi-modal data, improving model generalization and performance across diverse data distributions. Extensive experiments demonstrate the efficiency and effectiveness of PFMH within the federated learning framework, enabling distributed training for secure and precise multi-modal retrieval in real-world scenarios.Keywords
This publication has 16 references indexed in Scilit:
- Similarity Transitivity Broken-Aware Multi-Modal HashingIEEE Transactions on Knowledge and Data Engineering, 2024
- Multi-Modal Hashing for Efficient Multimedia Retrieval: A SurveyIEEE Transactions on Knowledge and Data Engineering, 2023
- FedProc: Prototypical contrastive federated learning on non-IID dataFuture Generation Computer Systems, 2023
- Partial Multi-Modal Hashing via Neighbor-Aware Completion LearningIEEE Transactions on Multimedia, 2023
- Unsupervised Adaptive Feature Selection With Binary HashingIEEE Transactions on Image Processing, 2023
- Flexible Discrete Multi-view Hashing with Collective Latent Feature LearningNeural Processing Letters, 2020
- Deep Collaborative Multi-View Hashing for Large-Scale Image SearchIEEE Transactions on Image Processing, 2020
- Efficient Supervised Discrete Multi-View Hashing for Large-Scale Multimedia SearchIEEE Transactions on Multimedia, 2019
- Fast Discrete Collaborative Multi-Modal Hashing for Large-Scale Multimedia RetrievalIEEE Transactions on Knowledge and Data Engineering, 2019
- Multiview Alignment Hashing for Efficient Image SearchIEEE Transactions on Image Processing, 2015