Capture Global Feature Statistics for One-Shot Federated Learning
Preprint
Abstract
Traditional Federated Learning (FL) necessitates numerous rounds of communication between the server and clients, posing significant challenges including high communication costs, connection drop risks and susceptibility to privacy attacks. One-shot FL has become a compelling learning paradigm to overcome above drawbacks by enabling the training of a global server model via a single communication round. However, existing one-shot FL methods suffer from expensive computation cost on the server or clients and cannot deal with non-IID (Independent and Identically Distributed) data stably and effectively. To address these challenges, this paper proposes FedCGS, a novel Federated learning algorithm that Capture Global feature Statistics leveraging pre-trained models. With global feature statistics, we achieve training-free and heterogeneity-resistant one-shot FL. Furthermore, we extend its application to personalization scenario, where clients only need execute one extra communication round with server to download global statistics. Extensive experimental results demonstrate the effectiveness of our methods across diverse data heterogeneity settings. Code is available at https://github.com/Yuqin-G/FedCGS.Keywords
All Related Versions
This publication has 11 references indexed in Scilit:
- FedDEO: Description-Enhanced One-Shot Federated Learning with Diffusion Models2024
- A Robust Privacy-Preserving Federated Learning Model Against Model Poisoning AttacksIEEE Transactions on Information Forensics and Security, 2024
- Privacy Inference Attack and Defense in Centralized and Federated Learning: A Comprehensive SurveyIEEE Transactions on Artificial Intelligence, 2024
- PromptFL: Let Federated Participants Cooperatively Learn Prompts Instead of Models – Federated Learning in Age of Foundation ModelIEEE Transactions on Mobile Computing, 2023
- Advances and Open Problems in Federated Learning2021
- Federated Learning: Challenges, Methods, and Future DirectionsIEEE Signal Processing Magazine, 2020
- BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding2019
- Practical Secure Aggregation for Privacy-Preserving Machine Learning2017
- Deep Residual Learning for Image Recognition2016
- A Bayesian Committee MachineNeural Computation, 2000