pFedES: Generalized Proxy Feature Extractor Sharing for Model Heterogeneous Personalized Federated Learning
Open Access
- 11 April 2025
- journal article
- Published by Association for the Advancement of Artificial Intelligence (AAAI) in Proceedings of the AAAI Conference on Artificial Intelligence
- Vol. 39 (21) , 22146-22154
Abstract
Federated learning (FL), as a privacy-preserving collaborative machine learning paradigm, has attracted significant interest from industry and academia. To allow each data owner (FL client) to train a heterogeneous and personalized local model based on its local data distribution, system resources and requirements on model structure, the field of model-heterogeneous personalized federated learning (MHPFL) has emerged. Existing MHPFL approaches either rely on the availability of a public dataset with special characteristics to facilitate knowledge transfer, incur high computational and communication costs, or face potential model leakage risks. To address these limitations, we propose a model-heterogeneous personalized Federated learning approach based on generalized proxy feature Extractor Sharing (pFedES) for supervised image classification tasks. (1) We devise a shared small proxy homogeneous feature extractor before each client's heterogeneous local model. (2) Clients train them via the proposed iterative learning to enable the exchange of global generalized knowledge and local personalized knowledge. (3) The small proxy local homogeneous extractors produced after local training are uploaded to the server for aggregation to facilitate knowledge fusion across clients. We theoretically prove pFedES converges with a non-convex convergence rate O(1/T). Experiments on 3 benchmark datasets against 9 baselines demonstrate that pFedES performs state-of-the-art model accuracy while maintaining efficient communication and computation.Keywords
This publication has 15 references indexed in Scilit:
- Decentralized federated learning through proxy model sharingNature Communications, 2023
- Towards Data-Independent Knowledge Transfer in Model-Heterogeneous Federated LearningIEEE Transactions on Computers, 2023
- Model Pruning Enables Efficient Federated Learning on Edge DevicesIEEE Transactions on Neural Networks and Learning Systems, 2022
- Communication-efficient federated learning via knowledge distillationNature Communications, 2022
- FedAUX: Leveraging Unlabeled Auxiliary Data in Federated LearningIEEE Transactions on Neural Networks and Learning Systems, 2021
- Federated learning on non-IID data: A surveyNeurocomputing, 2021
- Heterogeneous Model Fusion Federated Learning Mechanism Based on Model MappingIEEE Internet of Things Journal, 2021
- CFD: Communication-Efficient Federated Distillation via Soft-Label Quantization and Delta CodingIEEE Transactions on Network Science and Engineering, 2021
- Distillation-Based Semi-Supervised Federated Learning for Communication-Efficient Collaborative Training With Non-IID Private DataIEEE Transactions on Mobile Computing, 2021
- Gradient-based learning applied to document recognitionProceedings of the IEEE, 1998