Plug-and-Play Parameter-Efficient Tuning of Embeddings for Federated Recommendation
Preprint
Abstract
With the rise of cloud-edge collaboration, recommendation services are increasingly trained in distributed environments. Federated Recommendation (FR) enables such multi-end collaborative training while preserving privacy by sharing model parameters instead of raw data. However, the large number of parameters, primarily due to the massive item embeddings, significantly hampers communication efficiency. While existing studies mainly focus on improving the efficiency of FR models, they largely overlook the issue of embedding parameter overhead. To address this gap, we propose a FR training framework with Parameter-Efficient Fine-Tuning (PEFT) based embedding designed to reduce the volume of embedding parameters that need to be transmitted. Our approach offers a lightweight, plugin-style solution that can be seamlessly integrated into existing FR methods. In addition to incorporating common PEFT techniques such as LoRA and Hash-based encoding, we explore the use of Residual Quantized Variational Autoencoders (RQ-VAE) as a novel PEFT strategy within our framework. Extensive experiments across various FR model backbones and datasets demonstrate that our framework significantly reduces communication overhead while improving accuracy. The source code is available at https://github.com/young1010/FedPEFT.Keywords
All Related Versions
This publication has 8 references indexed in Scilit:
- Dual Test-Time Training for Out-of-Distribution Recommender SystemIEEE Transactions on Knowledge and Data Engineering, 2025
- Asynchronous Federated Learning With Local Differential Privacy for Privacy-Enhanced Recommender SystemsIEEE Internet of Things Journal, 2025
- PrivFR: Privacy-Enhanced Federated Recommendation With Shared Hash EmbeddingIEEE Transactions on Neural Networks and Learning Systems, 2024
- LightFR: Lightweight Federated Recommendation with Privacy-preserving Matrix FactorizationACM Transactions on Information Systems, 2022
- A federated graph neural network framework for privacy-preserving personalizationNature Communications, 2022
- Federated Neural Collaborative FilteringKnowledge-Based Systems, 2022
- Secure Federated Matrix FactorizationIEEE Intelligent Systems, 2020
- Federated Learning With Differential Privacy: Algorithms and Performance AnalysisIEEE Transactions on Information Forensics and Security, 2020