Heterogeneity-Aware Federated Learning with Adaptive Client Selection and Gradient Compression
Authors:
- 17 May 2023
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 1-10
Abstract
Federated learning (FL) allows multiple clients cooperatively train models without disclosing local data. However, the existing works fail to address all these practical concerns in FL: limited communication resources, dynamic network conditions and heterogeneous client properties, which slow down the convergence of FL. To tackle the above challenges, we propose a heterogeneity-aware FL framework, called FedCG, with adaptive client selection and gradient compression. Specifically, the parameter server (PS) selects a representative client subset considering statistical heterogeneity and sends the global model to them. After local training, these selected clients upload compressed model updates matching their capabilities to the PS for aggregation, which significantly alleviates the communication load and mitigates the straggler effect. We theoretically analyze the impact of both client selection and gradient compression on convergence performance. Guided by the derived convergence rate, we develop an iteration-based algorithm to jointly optimize client selection and compression ratio decision using submodular maximization and linear programming. Extensive experiments on both real-world prototypes and simulations show that FedCG can provide up to 5.3× speedup compared to other methods.Keywords
Funding Information
- Research and Development
- National Science Foundation
This publication has 25 references indexed in Scilit:
- Optimizing Federated Learning on Non-IID Data with Reinforcement Learning2020
- An Incentive Mechanism Design for Efficient Edge Learning by Deep Reinforcement Learning Approach2020
- TiFL: A Tier-based Federated Learning System2020
- Robust and Communication-Efficient Federated Learning From Non-i.i.d. DataIEEE Transactions on Neural Networks and Learning Systems, 2019
- A Convergence Analysis of Distributed SGD with Communication-Efficient Gradient Sparsification2019
- Client Selection for Federated Learning with Heterogeneous Resources in Mobile Edge2019
- Sparse Communication for Distributed Gradient Descent2017
- Accelerated greedy algorithms for maximizing submodular set functions2005
- An analysis of approximations for maximizing submodular set functions—IMathematical Programming, 1978
- On the Uncapacitated Location Problem1977