A Multi-Task Based Clustering Personalized Federated Learning Method
Authors:
- 1 December 2024
- journal article
- research article
- Published by Tsinghua University Press in Big Data Mining and Analytics
- Vol. 7 (4) , 1017-1030
Abstract
Federated Learning (FL) is a framework for machine learning on a large-scale distributed dataset, enabling the training of a collaborative model across multiple parties while preserving the privacy of user data. However, in cases where data are distributed in a non-independent and identically distributed (non-iid) manner, the convergence speed of the federated collaborative model and its prediction accuracy on client nodes can be significantly affected. Therefore, personalized FL methods have emerged to further adapt to the data characteristics of different clients. In response to the data heterogeneity issue, this paper presents a multi-task clustering-based personalized federated learning algorithm, which is applied to the prediction of carbon emissions in different regions and enterprises. This algorithm partitions nodes with similar data distributions and aggregates local models within the same cluster to form cluster models. It introduces the concept of multi-task learning, dividing the lower layers of cluster models into expert layers. These expert layers of different cluster models are then weighted and aggregated for the acquisition of global knowledge. Additionally, adaptive weight is applied to control the aggregation of expert layers, thereby achieving model personalization at the local level. Simulation experiments conducted on carbon emission prediction data demonstrate that the proposed algorithm performs better in various evaluation metrics compared with the Federated Averaging (FedAvg) algorithm and traditional clustering personalized federated learning algorithm. It also exhibits excellent experimental results and performance when dealing with different quantities of heterogeneous data distributions.Keywords
Funding Information
- National Key R&D Program of China (2022YFB2703400)
This publication has 17 references indexed in Scilit:
- Personalized federated learning: A Clustered Distributed Co-Meta-Learning approachInformation Sciences, 2023
- Multi-center federated learning: clients clustering for better personalizationWorld Wide Web, 2022
- Communication-efficient federated learning via knowledge distillationNature Communications, 2022
- Federated learning on non-IID data: A surveyNeurocomputing, 2021
- Personalized Federated Learning with Clustering: Non-IID Heart Rate Variability Data Application2021
- Federated learning with hierarchical clustering of local updates to improve training on non-IID data2020
- A Secure Federated Transfer Learning FrameworkIEEE Intelligent Systems, 2020
- FedHealth: A Federated Transfer Learning Framework for Wearable HealthcareIEEE Intelligent Systems, 2020
- CO2 emission sources, greenhouse gases, and the global warming effect2020
- The survey of economic growth, energy consumption and carbon emissionEnergy Reports, 2019