Boosting Gradient-based Training Diagnosis for Efficient and Accurate Federated Learning
- 12 March 2026
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Mobile Computing
- Vol. PP (15361233) , 1-16
Abstract
Federated Learning (FL) allows edge clients to collaborate in model training with data privacy preserved, yet it is known to suffer low training efficiency and model accuracy. Given that efficiency and accuracy are usually conflicting objectives, existing practices increasingly employ an adaptive scheme that changes the FL configurations (e.g., quantization or sparsification level) based on runtime training status, for which accurate training diagnosis—used for guiding the optimization actions—is crucial. However, while training diagnosis is a common task shared by different optimization schemes, existing works propose their diagnosis methods in an ad-hoc manner, which yield multiple limitations. First, the diagnosis metric in an optimization scheme may sometimes be less accurate than others; second, existing schemes fail to fully exploit the diagnosis result by applying it for only one optimization action; third, existing methods usually do not perceive cross-client data heterogeneity, failing to simultaneously enhance FL accuracy. To tackle those limitations, we make a systematical study on the training diagnosis methods of multiple optimization schemes, and propose metric grafting—replacing a scheme's diagnosis metric with a better one to improve the training performance. Moreover, to fully exploit the potential of training diagnosis, we build a system platform that supports flexible combinations of training diagnosis and optimization actions (i.e., single-diagnosis-multiple actions and multiple-diagnosis-multiple-actions). Evaluation on testbeds show that, with metric grafting and advanced diagnosis action combinations, we can substantially improve the efficiency and accuracy performance of FL.Keywords
Funding Information
- National Natural Science Foundation of China (62202300)
This publication has 21 references indexed in Scilit:
- AdaCoOpt: Leverage the Interplay of Batch Size and Aggregation Frequency for Federated Learning2023
- Communication-Efficient Federated Learning for Heterogeneous Edge Devices Based on Adaptive Gradient Quantization2023
- Top-k sparsification with secure aggregation for privacy-preserving federated learningComputers & Security, 2022
- GRACE: A Compressed Communication Framework for Distributed Machine Learning2021
- Cost-Effective Federated Learning Design2021
- Accelerating Distributed Deep Learning By Adaptive Gradient Quantization2020
- Adaptive Federated Learning in Resource Constrained Edge Computing SystemsIEEE Journal on Selected Areas in Communications, 2019
- EMNIST: Extending MNIST to handwritten letters2017
- Leveraging the error resilience of machine-learning applications for designing highly energy efficient accelerators2014
- Non-asymptotic confidence bounds for stochastic approximation algorithms with constant step sizeMonatshefte für Mathematik, 1990