Cepe-FL: Communication-Efficient and Privacy-Enhanced Federated Learning Via Adaptive Compressive Sensing
- 20 May 2024
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Big Data
- Vol. 11 (5) , 2119-2134
Abstract
Model updates are exchanged between server(s) and participants in Federated Learning (FL), which can result in excessive delay, especially for large models. Existing communication-efficient FL approaches such as quantization and top-k sampling apply compression to gradients assuming that gradients are sparse and can tolerate small deviations. This can hardly be applied to down-link transmission. In this work, we employ compressive sensing on model parameters instead of gradients and propose a two-way adaptive compression scheme, Cepe-FL, which exploits dictionary learning to project non-sparse model parameters into sparse representations to ensure reconstruction accuracy. Cepe-FL supports joint model reconstruction with drastic reduction in computational complexity from $O(n)$ to $O(1)$ . Cepe-FL adjusts the compression ratio adaptively according to the training loss, achieving the best trade-off between communication and model precision. Furthermore, it demonstrates efficacy in defending against membership inference attacks since only compressed models are exchanged. We conduct extensive experiments on three image classification tasks and compare with three communication-efficient approaches including FedPAQ, FedAvg and T-FedAvg. Cepe-FL presents the best performance in all tasks under IID and non-IID scenarios. We also implement white-box membership inference attacks, and the results show Cepe-FL can significantly suppress success ratio of inference in comparison with other approaches.
Keywords
Funding Information
- National Natural Science Foundation of China (61972081)
- Natural Science Foundation of Shanghai Municipality (22ZR1400200)
- Fundamental Research Funds for the Central Universities (2232023Y-01)
This publication has 33 references indexed in Scilit:
- Enhance membership inference attacks in federated learningComputers & Security, 2023
- FedDM: Iterative Distribution Matching for Communication-Efficient Federated Learning2023
- Communication-Efficient Federated Learning with Channel-Aware Sparsification over Wireless Networks2023
- CS-MIA: Membership inference attack based on prediction confidence series in federated learningJournal of Information Security and Applications, 2022
- Adaptive Quantization of Model Updates for Communication-Efficient Federated Learning2021
- Deep Leakage from Gradients2020
- Adaptive Gradient Sparsification for Efficient Federated Learning: An Online Learning Approach2020
- The restricted isometry property and its implications for compressed sensingComptes Rendus Mathématique, 2008
- Sparsity and incoherence in compressive samplingInverse Problems, 2007
- Extensions of compressed sensingSignal Processing, 2006