Time-Correlated Sparsification for Efficient Over-the-Air Model Aggregation in Wireless Federated Learning
Authors:
- 16 May 2022
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 3388-3393
Abstract
Federated edge learning (FEEL) is a promising distributed machine learning (ML) framework to drive edge intelligence applications. However, due to the dynamic wireless environments and the resource limitations of edge devices, communication becomes a major bottleneck. In this work, we propose time-correlated sparsification with hybrid aggregation (TCS-H) for communication-efficient FEEL, which exploits jointly the power of model compression and over-the-air computation. By exploiting the temporal correlations among model parameters, we construct a global sparsification mask, which is identical across devices, and thus enables efficient model aggregation over-the-air. Each device further constructs a local sparse vector to explore its own important parameters, which are aggregated via digital communication with orthogonal multiple access. We further design device scheduling and power allocation algorithms for TCS-H. Experiment results show that, under limited communication resources, TCS-H can achieve significantly higher accuracy compared to the conventional top-K sparsification with orthogonal model aggregation, with both i.i.d. and non-i.i.d. data distributions.Keywords
All Related Versions
Funding Information
- National Natural Science Foundation of China
- China Postdoctoral Science Foundation
- European Research Council
- CHIST-ERA
This publication has 13 references indexed in Scilit:
- Edge Learning with Timeliness Constraints: Challenges and SolutionsIEEE Communications Magazine, 2020
- Joint Device Scheduling and Resource Allocation for Latency Constrained Wireless Federated LearningIEEE Transactions on Wireless Communications, 2020
- Machine Learning at the Wireless Edge: Distributed Stochastic Gradient Descent Over-the-AirIEEE Transactions on Signal Processing, 2020
- Federated Learning Over Wireless Fading ChannelsIEEE Transactions on Wireless Communications, 2020
- Broadband Analog Aggregation for Low-Latency Federated Edge LearningIEEE Transactions on Wireless Communications, 2019
- Wireless Network Intelligence at the EdgeProceedings of the IEEE, 2019
- Scheduling Policies for Federated Learning in Wireless NetworksIEEE Transactions on Communications, 2019
- Joint scheduling and resource allocation in uplink OFDM systems for broadband wireless access networksIEEE Journal on Selected Areas in Communications, 2009
- Efficient subcarrier, power, and rate allocation with fairness consideration for OFDMA uplinkIEEE Transactions on Wireless Communications, 2008
- Technical Note—An Improved Algorithm for the Bottleneck Assignment ProblemOperations Research, 1971