Towards Efficient Asynchronous Federated Learning in Heterogeneous Edge Environments
Authors:
- 20 May 2024
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 2448-2457
Abstract
Federated learning (FL) is widely used in edge environments as a privacy-preserving collaborative learning paradigm. However, edge devices often have heterogeneous computation capabilities and data distributions, hampering the efficiency of co-training. Existing works develop staleness-aware semi-asynchronous FL that reduces the contribution of slow devices to the global model to mitigate their negative impacts. But this makes data on slow devices unable to be fully leveraged in global model updating, exacerbating the effects of data heterogeneity. In this paper, to cope with both system and data heterogeneity, we propose a clustering and two-stage aggregation-based Efficient Asynchronous Federated Learning (EAFL) framework, which can achieve better learning performance with higher efficiency in heterogeneous edge environments. In EAFL, we first propose a gradient similarity-based dynamic clustering mechanism to cluster devices with similar system and data characteristics together dynamically during the training process. Then, we develop a novel two-stage aggregation strategy consisting of staleness-aware semi-asynchronous intra-cluster aggregation and data size-aware synchronous inter-cluster aggregation to efficiently and comprehensively aggregate training updates across heterogeneous clusters. With that, the negative impacts of slow devices and Non-IID data can be simultaneously alleviated, thus achieving efficient collaborative learning. Extensive experiments demonstrate that EAFL is superior to state-of-the-art methods.Keywords
Funding Information
- National Natural Science Foundation of China
- Natural Science Foundation of Hunan Province
This publication has 29 references indexed in Scilit:
- Federated Learning with GAN-Based Data Synthesis for Non-IID Clients2023
- Federated Intrusion Detection on Non-IID Data for IIoT Networks Using Generative Adversarial Networks and Reinforcement Learning2022
- Byzantine-robust Federated Learning through Collaborative Malicious Gradient Filtering2022
- Federated Learning on Non-IID Data Silos: An Experimental Study2022
- A framework for privacy-preservation of IoT healthcare data using Federated Learning and blockchain technologyFuture Generation Computer Systems, 2022
- Federated learning on non-IID data: A surveyNeurocomputing, 2021
- Communication-efficient asynchronous federated learning in resource-constrained edge computingComputer Networks, 2021
- HySync: Hybrid Federated Learning with Effective Synchronization2020
- Hybrid-FL for Wireless Networks: Cooperative Learning Mechanism Using Non-IID Data2020
- Gradient-based learning applied to document recognitionProceedings of the IEEE, 1998