Secure Aggregation with Heterogeneous Quantization in Federated Learning
Preprint
Abstract
Secure model aggregation across many users is a key component of federated learning systems. The state-of-the-art protocols for secure model aggregation, which are based on additive masking, require all users to quantize their model updates to the same level of quantization. This severely degrades their performance due to lack of adaptation to available bandwidth at different users. We propose three schemes that allow secure model aggregation while using heterogeneous quantization. This enables the users to adjust their quantization proportional to their available bandwidth, which can provide a substantially better trade-off between the accuracy of training and the communication time. The proposed schemes are based on a grouping strategy by partitioning the network into groups, and partitioning the local model updates of users into segments. Instead of applying aggregation protocol to the entire local model update vector, it is applied on segments with specific coordination between users. We theoretically evaluate the quantization error for our schemes, and also demonstrate how our schemes can be utilized to overcome Byzantine users.Keywords
All Related Versions
This publication has 11 references indexed in Scilit:
- Information Theoretic Secure Aggregation With User DropoutsIEEE Transactions on Information Theory, 2022
- Dynamic Sampling and Selective Masking for Communication-Efficient Federated LearningIEEE Intelligent Systems, 2021
- Advances and Open Problems in Federated LearningFoundations and Trends in Machine Learning, 2021
- See through Gradients: Image Batch Recovery via GradInversion2021
- Turbo-Aggregate: Breaking the Quadratic Aggregation Barrier in Secure Federated LearningIEEE Journal on Selected Areas in Information Theory, 2021
- Shielding Collaborative Learning: Mitigating Poisoning Attacks through Client-Side DetectionIEEE Transactions on Dependable and Secure Computing, 2020
- Robust and Communication-Efficient Federated Learning From Non-i.i.d. DataIEEE Transactions on Neural Networks and Learning Systems, 2019
- 1-bit stochastic gradient descent and its application to data-parallel distributed training of speech DNNs2014
- The Byzantine Generals ProblemACM Transactions on Programming Languages and Systems, 1982
- New directions in cryptographyIEEE Transactions on Information Theory, 1976