PFedKD: Personalized Federated Learning via Knowledge Distillation Using Unlabeled Pseudo Data for Internet of Things
- 28 January 2025
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Internet of Things Journal
- Vol. 12 (11) , 16314-16324
Abstract
With the rapid advancement of wearable devices and IoT technologies, sensor data generated by edge devices has surged. This data is crucial for advancing IoT applications, including health status monitoring, abnormal behavior detection, and environmental monitoring. However, traditional centralized learning requires uploading data to a central server, raising security and privacy concerns and hindering data application. Federated learning (FL) offers a solution by enabling collaborative model training on IoT devices without transferring data from the local device. In practice, edge devices generate data that is often highly heterogeneous, making it challenging for the global FL model to capture local data distributions accurately, leading to significant performance degradation. Additionally, imbalanced edge device resources and limited bandwidth can cause data transmission delays or interruptions, impacting application feasibility. To address these issues, we propose PFedKD, a novel personalized FL algorithm based on knowledge distillation, aimed at enhancing the model’s generalization ability and reducing communication overhead in heterogeneous IoT data environments. PFedKD constructs a public dataset using unlabeled pseudo data to extract knowledge from each client, training personalized models that fit local data distributions. This method controls dataset size while enhancing performance. During communication, only logits and class prototypes are transmitted, ensuring high communication efficiency. Sharpness Aware Minimization is introduced in local model training to optimize generalization. Additionally, we design a weight distribution mechanism based on client sample quality evaluation that optimizes knowledge aggregation and model personalization. Extensive experiments demonstrate that PFedKD significantly outperforms state-of-the-art baselines in both learning performance and communication efficiency.Keywords
Funding Information
- National Natural Science Foundation of China (U21A20463, U22B2027, U23A20304)
- Fundamental Research Funds for the Central Universities (2024ZYCX014)
- Beijing Natural Science Foundation (L221014)
- Systematic Major Project of China State Railway Group Corporation Limited (P2023W002)
- Hangzhou Qianjiang Distinguished Experts Programme 2024
This publication has 25 references indexed in Scilit:
- CoBA: Collusive Backdoor Attacks With Optimized Trigger to Federated LearningIEEE Transactions on Dependable and Secure Computing, 2024
- Distributed Foundation Models for Multi-Modal Learning in 6G Wireless NetworksIEEE Wireless Communications, 2024
- Adaptive Federated Learning With Negative Inner Product AggregationIEEE Internet of Things Journal, 2024
- Personalized Federated Learning With Model-Contrastive Learning for Multi-Modal User Modeling in Human-Centric MetaverseIEEE Journal on Selected Areas in Communications, 2024
- Privacy-Preserving Federated Learning With Resource-Adaptive Compression for Edge DevicesIEEE Internet of Things Journal, 2023
- Federated Learning on Non-IID Data Silos: An Experimental Study2022
- Federated Learning for Industrial Internet of Things in Future IndustriesIEEE Wireless Communications, 2021
- Federated learning with hierarchical clustering of local updates to improve training on non-IID data2020
- Deep Residual Learning for Image Recognition2016
- Gradient-based learning applied to document recognitionProceedings of the IEEE, 1998