Heterogeneous Federated Learning via Knowledge Transfer Guided by Global Pseudo Proxy Data
Open Access
- 8 January 2026
- journal article
- research article
- Published by MDPI AG in Future Internet
- Vol. 18 (1) , 36
Abstract
Federated learning with data free knowledge distillation enables effective and privacy-preserving knowledge aggregation by employing generators to produce local pseudo samples during client-side model migration. However, in practical applications, data distributions across different institutions are often non-independent and identically distributed (Non-IID), which introduces bias in local models and consequently impedes the effective transfer of knowledge to the global model. In addition, insufficient local training can further exacerbate model bias, undermining overall performance. To address these challenges, we propose a heterogeneous federated learning framework that enhances knowledge transfer through guidance from global proxy data. Specifically, a noise filter is incorporated into the training of local generators to mitigate the negative impact of low-quality pseudo proxy samples on local knowledge distillation. Furthermore, a global generator is introduced to produce global pseudo proxy samples, which, together with local pseudo proxy data, are used to construct a cross attention matrix. This design effectively alleviates overfitting and underfitting issues in local models caused by data heterogeneity. Extensive experiments on publicly available datasets with heterogeneous data distributions demonstrate the superiority of the proposed framework. Results show that when the Dirichlet distribution coefficient is 0.05, our method achieves an average accuracy improvement of 5.77% over popular baselines; when the coefficient is 0.1, the improvement reaches 6.54%. Even under uniformly distributed sample classes, our model still achieves an average accuracy improvement of 7.07% compared to other methods.Keywords
Funding Information
- National Natural Science Foundation of China Youth Fund (61902165)
- Liaoning Provincial Department of Education Fund (203070132279)
- Liaoning Provincial Department of Education Fund (JYTMS2023104)
This publication has 24 references indexed in Scilit:
- Emerging AI threats in cybercrime: a review of zero-day attacks via machine, deep, and federated learningKnowledge and Information Systems, 2025
- Enhanced Model Poisoning Attack and Multi-Strategy Defense in Federated LearningIEEE Transactions on Information Forensics and Security, 2025
- Data-free knowledge distillation via generator-free data generation for Non-IID federated learningNeural Networks, 2024
- Privacy-Centric AI: Navigating the Landscape with Federated LearningInternational Journal for Research in Applied Science and Engineering Technology, 2024
- Exploring the Distributed Knowledge Congruence in Proxy-data-free Federated DistillationACM Transactions on Intelligent Systems and Technology, 2024
- FedProc: Prototypical contrastive federated learning on non-IID dataFuture Generation Computer Systems, 2023
- Safe: Synergic Data Filtering for Federated Learning in Cloud-Edge ComputingIEEE Transactions on Industrial Informatics, 2022
- Federated Learning in Mobile Edge Networks: A Comprehensive SurveyIEEE Communications Surveys & Tutorials, 2020
- Heterogeneous Data and Big Data AnalyticsAutomatic Control and Information Sciences, 2017
- On Definition and Quantification of HeterogeneityOikos, 1995