Enhancing Decentralized Federated Learning With Model Pruning and Adaptive Communication
- 18 September 2024
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Industrial Informatics
- Vol. 21 (1) , 70-84
Abstract
Federated learning (FL) is a distributed learning paradigm that enables large-scale IoT devices to collaboratively train a shared model while preserving the privacy of local data. To avoid the single-point-of-failure of the conventional parameter server architecture, the study concentrates on the decentralized FL (DFL) paradigm building on the device-to-device communication network. However, existing DFL frameworks encounter challenges related to resource limitations, privacy protection, and data heterogeneity. To overcome these challenges, the study proposes and implements DF $^{2}$ -MPC in industrial IoT, an efficient DFL framework with personalized model pruning and adaptive communication. Specifically, a personalized pruning ratio determination approach is designed by exploiting the model pruning technique. This approach enables all devices to flexibly determine pruning ratios by themselves, thereby achieving both communication savings and privacy protection. Then, this study designs an adaptive neighbor selection scheme, which can enhance model performance and foster model consensus under resource constraints. In addition, the study theoretically proves the convergence performance of DF $^{2}$ -MPC. Finally, extensive simulations on three real-world traces are conducted to corroborate the superiority of DF $^{2}$ -MPC, demonstrating that the method can improve communication efficiency with satisfactory model accuracy and convergence performance.Keywords
Funding Information
- National Natural Science Foundation of China (62172386, 62436010, 62472298, 62102275, 61872330, 61572457)
- Natural Science Foundation of Jiangsu Province (BK20231212, BK20210704, BK20191194)
- National Natural Science Foundation of USA (CNS 2128378, CNS 2107014, CNS 1824440, CNS 1828363)
- China Postdoctoral Science Foundation (2023M732537)
This publication has 35 references indexed in Scilit:
- Energy-efficient distributed federated learning offloading and scheduling healthcare system in blockchain based networksInternet of Things, 2023
- Optimal Rate Adaption in Federated Learning with Compressed Communications2022
- Federated-Learning Based Privacy Preservation and Fraud-Enabled Blockchain IoMT System for HealthcareIEEE Journal of Biomedical and Health Informatics, 2022
- Federated Learning-Aware Multi-Objective Modeling and blockchain-enable system for IIoT applicationsComputers and Electrical Engineering, 2022
- Decentralized Machine Learning Through Experience-Driven Method in Edge NetworksIEEE Journal on Selected Areas in Communications, 2021
- Semi-Decentralized Federated Learning With Cooperative D2D Local Model AggregationsIEEE Journal on Selected Areas in Communications, 2021
- Communication-efficient Decentralized Machine Learning over Heterogeneous Networks2021
- Communication-Efficient Decentralized Learning with Sparsification and Adaptive Peer Selection2020
- PPCL: Privacy-preserving collaborative learning for mitigating indirect information leakageInformation Sciences, 2020
- Importance Estimation for Neural Network Pruning2019