Latency-Efficient Wireless Federated Learning With Sparsification and Quantization for Heterogeneous Devices
- 17 September 2024
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Internet of Things Journal
- Vol. 12 (1) , 488-501
Abstract
Recently, federated learning (FL) has attracted much attention as a promising decentralized machine learning method that provides privacy and low latency. However, the communication bottleneck is still a problem that needs to be solved to effectively deploy FL on wireless networks. In this paper, we aim to minimize the total convergence time of FL by sparsifying and quantizing local model parameters before uplink transmission. More specifically, we first present the convergence analysis of the FL algorithm with random sparsification and quantization, revealing the impact of compression error on the convergence speed. Then, we jointly optimize the computation, communication resources and the number of quantization bits, sparsity to minimize the total convergence time, subject to the energy and compression error requirements derived from the convergence analysis. By simulating the impact of different compression errors on model accuracy, we reveal that the low-precision updates do not inherently yield a better balance between efficiency and accuracy than the high-precision updates. Furthermore, compared with the equal resource allocation schemes and the unilateral compression optimization schemes on four different data distributions, the proposed scheme has faster convergence speed and less total convergence time.Keywords
Funding Information
- Joint Funds for Railway Fundamental Research of National Natural Science Foundation of China (U2368201)
- NSF of China (62172441)
- Key Research and Development Program of Sichuan Province (2023YFG0120)
This publication has 26 references indexed in Scilit:
- Latency Minimization for Wireless Federated Learning With Heterogeneous Local Model UpdatesIEEE Internet of Things Journal, 2023
- Federated Learning Over Wireless Networks: Challenges and SolutionsIEEE Internet of Things Journal, 2023
- Time-Correlated Sparsification for Efficient Over-the-Air Model Aggregation in Wireless Federated Learning2022
- Jointly Optimizing Client Selection and Resource Management in Wireless Federated Learning for Internet of ThingsIEEE Internet of Things Journal, 2021
- A Survey on Federated Learning for Resource-Constrained IoT DevicesIEEE Internet of Things Journal, 2021
- To Talk or to Work: Flexible Communication Compression for Energy Efficient Federated Learning over Heterogeneous Mobile Edge Devices2021
- Design and Analysis of Uplink and Downlink Communications for Federated LearningIEEE Journal on Selected Areas in Communications, 2020
- Device Scheduling with Fast Convergence for Wireless Federated Learning2020
- Deep Learning With Edge Computing: A ReviewProceedings of the IEEE, 2019
- A Distributed Synchronous SGD Algorithm with Global Top-k Sparsification for Low Bandwidth Networks2019