Fed-RAC: Resource-Aware Clustering for Tackling Heterogeneity of Participants in Federated Learning
- 20 March 2024
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Parallel and Distributed Systems
- Vol. 35 (7) , 1207-1220
Abstract
Federated Learning is a training framework that enables multiple participants to collaboratively train a shared model while preserving data privacy. The heterogeneity of devices and networking resources of the participants delay the training and aggregation. The paper introduces a novel approach to federated learning by incorporating resource-aware clustering. This method addresses the challenges posed by the diverse devices and networking resources among participants. Unlike static clustering approaches, this paper proposes a dynamic method to determine the optimal number of clusters using Dunn Indices. It enables adaptability to the varying heterogeneity levels among participants, ensuring a responsive and customized approach to clustering. Next, the paper goes beyond empirical observations by providing a mathematical derivation of the communication rounds for convergence within each cluster. Further, the participant assignment mechanism adds a layer of sophistication and ensures that devices and networking resources are allocated optimally. Afterwards, we incorporate a master-slave technique, particularly through knowledge distillation, which improves the performance of lightweight models within clusters. Finally, experiments are conducted to validate the approach and to compare it with state-of-the-art. The results demonstrated an accuracy improvement of over 3% compared to its closest competitor and a reduction in communication rounds of around 10%.Keywords
Funding Information
- I-DAPT HUB Foundation
- Anusandhan National Research Foundation (l-DAPT/IIT (BHU)/2023-24/ Project Sanction/43, MTR/2023/000764)
- NSF (CNS-2008878)
- Cyberinfrastructure for Accelerating Innovation in Network Dynamics (OAC-2104078)
This publication has 22 references indexed in Scilit:
- FedSA: A Semi-Asynchronous Federated Learning Mechanism in Heterogeneous Edge ComputingIEEE Journal on Selected Areas in Communications, 2021
- SHARE: Shaping Data Distribution at Edge for Communication-Efficient Hierarchical Federated Learning2021
- An Asynchronous and Real-Time Update Paradigm of Federated Learning for Fault DiagnosisIEEE Transactions on Industrial Informatics, 2021
- Highlight Every Step: Knowledge Distillation via Collaborative TeachingIEEE Transactions on Cybernetics, 2020
- Federated learning with hierarchical clustering of local updates to improve training on non-IID data2020
- Communication-Efficient Federated Deep Learning With Layerwise Asynchronous Model Update and Temporally Weighted AggregationIEEE Transactions on Neural Networks and Learning Systems, 2019
- Adaptive Federated Learning in Resource Constrained Edge Computing SystemsIEEE Journal on Selected Areas in Communications, 2019
- An improved DBSCAN, a density based clustering algorithm with parameter selection for high dimensional data sets2012
- Some new indexes of cluster validityIEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics), 1998
- Gradient-based learning applied to document recognitionProceedings of the IEEE, 1998