Personalized Federated Learning on Non-IID Data via Group-based Meta-learning
Open Access
- 22 March 2023
- journal article
- research article
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Knowledge Discovery From Data
- Vol. 17 (4) , 1-20
Abstract
Personalized federated learning (PFL) has emerged as a paradigm to provide a personalized model that can fit the local data distribution of each client. One natural choice for PFL is to leverage the fast adaptation capability of meta-learning, where it first obtains a single global model, and each client achieves a personalized model by fine-tuning the global one with its local data. However, existing meta-learning-based approaches implicitly assume that the data distribution among different clients is similar, which may not be applicable due to the property of data heterogeneity in federated learning. In this work, we propose a Group-based Federated Meta-Learning framework, called G-FML , which adaptively divides the clients into groups based on the similarity of their data distribution, and the personalized models are obtained with meta-learning within each group. In particular, we develop a simple yet effective grouping mechanism to adaptively partition the clients into multiple groups. Our mechanism ensures that each group is formed by the clients with similar data distribution such that the group-wise meta-model can achieve “personalization” at large. By doing so, our framework can be generalized to a highly heterogeneous environment. We evaluate the effectiveness of our proposed G-FML framework on three heterogeneous benchmarking datasets. The experimental results show that our framework improves the model accuracy by up to 13.15% relative to the state-of-the-art federated meta-learning.Keywords
Funding Information
- National Natural Science Foundation of China (61972161)
- Guangdong Basic and Applied Basic Research Foundation (2022A1515010374)
- Hong Kong RGC General Research Fund (PolyU 15220020 and PolyU 15204921)
- Hong Kong RGC Research Impact Fund (R5060-19)
- Hong Kong RGC Theme-Based Research Scheme (T41-603/20-R)
This publication has 11 references indexed in Scilit:
- A Federated Graph Neural Network Framework for Privacy-Preserving PersonalizationResearch Square, 2022
- FedGroup: Efficient Federated Learning via Decomposed Similarity-Based Clustering2021
- Advances and Open Problems in Federated LearningFoundations and Trends in Machine Learning, 2021
- A survey on security and privacy of federated learningFuture Generation Computer Systems, 2021
- Clustered Federated Learning: Model-Agnostic Distributed Multitask Optimization Under Privacy ConstraintsIEEE Transactions on Neural Networks and Learning Systems, 2020
- Self-Balancing Federated Learning With Global Imbalanced Data in Mobile SystemsIEEE Transactions on Parallel and Distributed Systems, 2020
- Federated Learning: Challenges, Methods, and Future DirectionsIEEE Signal Processing Magazine, 2020
- FedHealth: A Federated Transfer Learning Framework for Wearable HealthcareIEEE Intelligent Systems, 2020
- Federated Machine LearningACM Transactions on Intelligent Systems and Technology, 2019
- EMNIST: Extending MNIST to handwritten letters2017