Communication-Efficient Federated Learning Over Capacity-Limited Wireless Networks
- 25 June 2024
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Cognitive Communications and Networking
- Vol. 11 (1) , 621-637
Abstract
In this paper, we propose a communication-efficient federated learning (FL) framework to enhance the convergence rate of FL under limited uplink capacity. The core idea of our framework is to transmit the values and positions of the top-S entries of a local model update, determined in terms of magnitude. When transmitting the top-S values, we first apply a linear transformation that enforces the transformed values to behave like Gaussian random variables. We then employ a scalar quantizer optimized for Gaussian distributions, leading to minimizing compression errors. When reconstructing the top-S values, we develop a linear minimum mean squared error method based on the Bussgang decomposition. Additionally, we introduce an error feedback strategy to compensate for both compression and reconstruction errors. We analyze the convergence rate of our framework under general considerations, including a non-convex loss function. Based on our analytical results, we optimize the key parameters of our framework to maximize the convergence rate for a given uplink capacity. Simulation results demonstrate that our framework achieves more than a 2.2%, 1.1%, and 1.4% increase in classification accuracy for the MNIST, CIFAR-10, and CIFAR-100 datasets, respectively, compared to state-of-the-art FL frameworks under limited uplink capacity.Keywords
All Related Versions
Funding Information
- National Research Foundation of Korea
- Korea government (2022R1C1C1010074)
- Institute of Information & Communications Technology Planning & Evaluation
- Korea government (2021-0-00972)
- Development of Intelligent Wireless Access Technology, [50%])
- BK21 FOUR Program
- National Science Foundation (EECS-2335876)
This publication has 33 references indexed in Scilit:
- Federated Learning Over Wireless IoT Networks With Optimized Communication and ResourcesIEEE Internet of Things Journal, 2022
- Time-Correlated Sparsification for Communication-Efficient Federated Learning2021
- Communication-efficient federated learningProceedings of the National Academy of Sciences, 2021
- Communication-Efficient Federated Learning Based on Compressed SensingIEEE Internet of Things Journal, 2021
- Sparse Binary Compression: Towards Distributed Deep Learning with minimal Communication2019
- ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devices2018
- Deep Residual Learning for Image Recognition2016
- An empirically based path loss model for wireless channels in suburban environmentsIEEE Journal on Selected Areas in Communications, 1999
- Gradient-based learning applied to document recognitionProceedings of the IEEE, 1998
- Lexicographic ordering, ranking and unranking of combinationsInternational Journal of Computer Mathematics, 1985