FedAS: Bridging Inconsistency in Personalized Federated Learning
Authors:
- 16 June 2024
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 11986-11995
Abstract
Personalized Federated Learning (PFL) is primarily de-signed to provide customized models for each client to better fit the non-iid distributed client data, which is a inherent challenge in Federated Learning. However, current PFL methods suffer from inconsistencies in both intra-client and inter-client levels: 1) The intra-client inconsistency stems from the asynchronous update strategy for personalized and shared parameters. In PFL, clients update their shared parameters to communicate and learn from others, while keeping personalized parts unchanged, leading to poor coordination between these two components. 2) The Inter-client inconsistency arises from “stragglers” - inactive clients that communicate and train with the server less frequently. This results in their under-trained personalized models and impedes the collaborative training stage for other clients. In this paper, we present a novel PFL framework named FedAS, which uses Federated Parameter-Alignment and Client-Synchronization to overcome above challenges. Initially, we enhance the localization of global parameters by infusing them with local insights. We make the shared parts learn from previous model, thereby increasing their local relevance and reducing the impact of parameter inconsistency. Furthermore, we design a robust aggregation method to mitigate the impact of stragglers by preventing the incorporation of their under-trained knowledge into aggregated model. Experimental results on CifarlO and CifarlOO validate the effectiveness of our FedAS in achieving better performance and robustness against data heterogeneity.Keywords
This publication has 36 references indexed in Scilit:
- FedIIC: Towards Robust Federated Learning for Class-Imbalanced Medical Image Classification2023
- FedProc: Prototypical contrastive federated learning on non-IID dataFuture Generation Computer Systems, 2023
- Local Learning Matters: Rethinking Data Heterogeneity in Federated Learning2022
- CD2-pFed: Cyclic Distillation-guided Channel Decoupling for Model Personalization in Federated Learning2022
- Robust Federated Learning with Noisy and Heterogeneous Clients2022
- FedDC: Federated Learning with Non-IID Data via Local Drift Decoupling and Correction2022
- Model-Contrastive Federated Learning2021
- A Tutorial on Fisher informationJournal of Mathematical Psychology, 2017
- Overcoming catastrophic forgetting in neural networksProceedings of the National Academy of Sciences, 2017
- Layer-Wise Relevance Propagation for Neural Networks with Local Renormalization Layers2016