Multi-Metric Client Activation Method for Fast and Accurate Federated Learning
Open Access
- 19 March 2026
- journal article
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Intelligent Systems and Technology
- Vol. 17 (2) , 1-27
Abstract
While unbiased gradient estimators ensure unbiased solutions in empirical risk minimization problems, they can significantly hinder optimization efficiency and generalization performance in strongly non-IID Federated Learning environments. Although recent studies have demonstrated promising applications of biased estimators, they typically focus only on convergence rates while overlooking generalization performance. We propose a novel multi-metric bias concept, quantified using both local loss and local gradient norm, along with a client activation method based on this bias concept. The proposed method prioritizes training on local datasets that better represent the global dataset, leading to faster convergence and improved generalization. Our extensive empirical study demonstrates that carefully injecting bias into client activation accelerates federated optimization, achieving a substantially improved validation accuracy within a given epoch budget. In representative machine learning benchmarks, our method achieves up to \(12.7\%\) higher accuracy than uniform random sampling and \(2.5\%\) higher accuracy than state-of-the-art biased client activation methods.
Keywords
Funding Information
- Institute of Information and Communications Technology Planning and Evaluation (IITP), grant funded by the Korea Government (RS-2022-00155915)
- Artificial Intelligence Convergence Innovation Human Resources Development
- National Research Foundation of Korea (NRF), grant funded by the Korea government (No. RS-2024-00452914)
This publication has 16 references indexed in Scilit:
- Advocating for the Silent: Enhancing Federated Generalization for Nonparticipating ClientsIEEE Transactions on Neural Networks and Learning Systems, 2024
- Partial model averaging in Federated Learning: Performance guarantees and benefitsNeurocomputing, 2023
- Gradient Norm Aware Minimization Seeks First-Order Flatness and Improves Generalization2023
- Generalization Matters: Loss Minima Flattening via Parameter Hybridization for Efficient Online Knowledge Distillation2023
- Achieving small-batch accuracy with large-batch scalability via Hessian-aware learning rate adjustmentNeural Networks, 2022
- Optimizing Federated Learning on Non-IID Data with Reinforcement Learning2020
- Improving Scalability of Parallel CNN Training by Adjusting Mini-Batch Size at Run-Time2019
- Deep Residual Learning for Image Recognition2016
- On the Behavior of the Gradient Norm in the Steepest Descent MethodComputational Optimization and Applications, 2002
- Nonlinear ProgrammingJournal of the Operational Research Society, 1997