Personalized Federated Learning with Layer-Wise Feature Transformation via Meta-Learning
Open Access
- 13 February 2024
- journal article
- research article
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Knowledge Discovery From Data
- Vol. 18 (4) , 1-21
Abstract
Federated learning enables multiple clients to collaboratively learn machine learning models in a privacy-preserving manner. However, in real-world scenarios, a key challenge encountered in federated learning is the statistical heterogeneity among clients. Existing work mainly focused on a single global model shared across the clients, making it hard to generalize well to all clients due to the large discrepancy in the data distributions. To address this challenge, we propose pFedLT , a novel approach that can adapt the single global model to different data distributions. Specifically, we propose to perform a pluggable layer-wise transformation during the local update phase based on scaling and shifting operations. In particular, these operations are learned with a meta-learning strategy. By doing so, pFedLT can capture the diversity of data distribution among clients, therefore, can generalize well even when the data distributions among clients exhibit high statistical heterogeneity. We conduct extensive experiments on synthetic and real-world datasets (MNIST, Fashion_MNIST, CIFAR-10, and Office+Caltech10) under different Non-IID settings. Experimental results demonstrate that pFedLT significantly improves the model accuracy by up to 11.67% and reduces the communication costs compared with state-of-the-art approaches.Keywords
Funding Information
- National Natural Science Foundation of China (61972161, and P0040473)
- Guangdong Basic and Applied Basic Research Foundation (2022A1515010374)
- Guangzhou Basic and Applied Basic Research Foundation (202201010715)
- Theme-Based Research Scheme (T43-513/23-N)
This publication has 18 references indexed in Scilit:
- Federated learning with adaptive communication compression under dynamic bandwidth and unreliable networksInformation Sciences, 2020
- Clustered Federated Learning: Model-Agnostic Distributed Multitask Optimization Under Privacy ConstraintsIEEE Transactions on Neural Networks and Learning Systems, 2020
- Federated learning with hierarchical clustering of local updates to improve training on non-IID data2020
- FedSteg: A Federated Transfer Learning Framework for Secure Image SteganalysisIEEE Transactions on Network Science and Engineering, 2020
- Federated Learning: Challenges, Methods, and Future DirectionsIEEE Signal Processing Magazine, 2020
- CoLearn2020
- In-Edge AI: Intelligentizing Mobile Edge Computing, Caching and Communication by Federated LearningIEEE Network, 2019
- Adaptive Federated Learning in Resource Constrained Edge Computing SystemsIEEE Journal on Selected Areas in Communications, 2019
- ImageNet classification with deep convolutional neural networksCommunications of the ACM, 2017
- Gradient-based learning applied to document recognitionProceedings of the IEEE, 1998