Federated Learning With Soft Clustering
- 21 September 2021
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Internet of Things Journal
- Vol. 9 (10) , 7773-7782
Abstract
In this article, we consider the problem of federated learning (FL) with training data that are non independent and identically distributed (non-IID) across the clients. To cope with data heterogeneity, an iterative federated clustering algorithm (IFCA) has been proposed. IFCA partitions the clients into a number of clusters and lets the clients in the same cluster optimize a shared model. However, in IFCA, the clusters are nonoverlapping, which leads to an inefficient utilization of the local information since the knowledge of a client is used by only one cluster during each round. To capture the complex nature of real-world data, soft clustering methods with overlapping clusters have been proposed that attain superior performance over the hard ones. Motivated by this, we propose a new algorithm named FL with soft clustering (FLSC) by combining the strengths of soft clustering and IFCA, where the clients are partitioned into overlapping clusters and the information of each participating client is used by multiple clusters simultaneously during each round. The experimental results show that FLSC achieves better learning performance on the classification tasks on the MNIST and Fashion-MNIST data sets, compared with the state-of-the-art baseline methods, i.e., the global model method and IFCA.Keywords
Funding Information
- National Natural Science Foundation of China (61790551, 61925106)
This publication has 28 references indexed in Scilit:
- Observing the Pulse of a City: A Smart City Framework for Real-Time Discovery, Federation, and Aggregation of Data StreamsIEEE Internet of Things Journal, 2018
- Non-Exhaustive, Overlapping ClusteringIEEE Transactions on Pattern Analysis and Machine Intelligence, 2018
- Semisupervised Deep Reinforcement Learning in Support of IoT and Smart City ServicesIEEE Internet of Things Journal, 2017
- Mobile Phone Computing and the Internet of Things: A SurveyIEEE Internet of Things Journal, 2016
- Clustering with Spectral Norm and the k-Means Algorithm2010
- Model-based overlapping clustering2005
- Optimal linear estimation fusion. I. Unified fusion rulesIEEE Transactions on Information Theory, 2003
- A modified version of the K-means algorithm with a distance based on cluster symmetryIEEE Transactions on Pattern Analysis and Machine Intelligence, 2001
- Gradient-based learning applied to document recognitionProceedings of the IEEE, 1998
- FCM: The fuzzy c-means clustering algorithmComputers & Geosciences, 1984