CyclicFL: Efficient Federated Learning with Cyclic Model Pre-training
- 5 February 2025
- journal article
- Published by World Scientific Pub Co Pte Ltd in Journal of Circuits, Systems and Computers
- Vol. 34 (07)
Abstract
Federated learning (FL) has been proposed to enable distributed learning on artificial intelligence Internet of Things (AIoT) devices with guarantees of high-level data privacy. Since random initial models in FL can easily result in unregulated stochastic gradient descent (SGD) processes, existing FL methods greatly suffer from both slow convergence and poor accuracy, especially in non-IID scenarios. To address this problem, we propose a novel method named CyclicFL, which can quickly derive effective initial models to guide the SGD processes, thus improving the overall FL training performance. We formally analyze the significance of data consistency between the pre-training and training stages of CyclicFL, showing the limited Lipschitzness of loss for the pre-trained models by CyclicFL. Moreover, we systematically prove that our method can achieve a faster convergence speed under various convexity assumptions. Unlike traditional centralized pre-training methods that require public proxy data, CyclicFL pre-trains initial models on selected AIoT devices cyclically without exposing their local data. Therefore, they can be easily integrated into any security-critical FL methods. Comprehensive experimental results show that CyclicFL can not only improve maximum classification accuracy by up to 14.11%, but also significantly accelerate the overall FL training process.Keywords
Funding Information
- Shanghai International Joint Lab of Trustworthy Intelligent Software (22510750100)
This publication has 18 references indexed in Scilit:
- Power consumption reduction for IoT devices thanks to Edge-AI: Application to human activity recognitionInternet of Things, 2023
- Efficient Federated Learning for AIoT Applications Using Knowledge DistillationIEEE Internet of Things Journal, 2022
- FedDC: Federated Learning with Non-IID Data via Local Drift Decoupling and Correction2022
- Federated Learning on Non-IID Data Silos: An Experimental Study2022
- Federated learning on non-IID data: A surveyNeurocomputing, 2021
- Model-Contrastive Federated Learning2021
- Deep Residual Learning for Image Recognition2016
- Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification2015
- Gradient-based learning applied to document recognitionProceedings of the IEEE, 1998
- Smoothing and Differentiation of Data by Simplified Least Squares Procedures.Analytical Chemistry, 1964