K-FL: Kalman Filter-Based Clustering Federated Learning Method
Open Access
- 5 April 2023
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 11, 36097-36105
Abstract
Federated learning is a distributed machine learning framework that enables a large number of devices to cooperatively train a model without data sharing. However, because federated learning trains a model using non-independent and identically distributed (non-IID) data stored at local devices, the weight divergence causes a performance loss. This paper focuses on solving the non-IID problems and proposes Kalman filter-based clustering federated learning method called K-FL to get performance gain by providing a specific model with low variance to the device. To the best of our knowledge, it is the first clustering federated learning method that can train a model requiring fewer communication rounds under the premise that non-IID environment without any prior knowledge and an initial value set by the user. From simulations, we demonstrate that the proposed K-FL can train a model much faster, requiring fewer communication rounds than FedAvg and LG-FedAvg when testing neural networks using the MNIST, FMNIST, and CIFAR-10 datasets. As a numerical result, it is shown that the accuracy is improved in all datasets while the computational time cost is reduced by 1.43×, 1.67×, and 1.63× compared to FedAvg, respectively.Keywords
Funding Information
- National Research Foundation of Korea (NRF) and Institute for Information & Communications Technology Planning & Evaluation
- Korean Government [Ministry of Science and ICT, South Korea (MSIT)]
- Acquisition Indication-Media Access Control (AI-MAC) Protocol on Distributed Machine Learning for Intelligent Flying Base Station (2022R1A2C2005705)
- Development of the 6G Service Targeted Artificial Intelligence (AI)/Machine Learning (ML)-Based Autonomous Regulating Medium Access Control (2021-0-00368)
This publication has 10 references indexed in Scilit:
- Reward-based participant selection for improving federated reinforcement learningICT Express, 2022
- Federated Learning on Non-IID Data Silos: An Experimental Study2022
- Information-theoretic privacy in federated submodel learningICT Express, 2022
- Federated learning on non-IID data: A surveyNeurocomputing, 2021
- Reducing Model Cost Based on the Weights of Each Layer for Federated Learning Clustering2021
- Advances and Open Problems in Federated LearningFoundations and Trends in Machine Learning, 2021
- Clustered Federated Learning: Model-Agnostic Distributed Multitask Optimization Under Privacy ConstraintsIEEE Transactions on Neural Networks and Learning Systems, 2020
- Federated learning with hierarchical clustering of local updates to improve training on non-IID data2020
- Privacy in the Internet of Things: threats and challengesSecurity and Communication Networks, 2013
- Gradient-based learning applied to document recognitionProceedings of the IEEE, 1998