Abstract
While unbiased gradient estimators ensure unbiased solutions in empirical risk minimization problems, they can significantly hinder optimization efficiency and generalization performance in strongly non-IID Federated Learning environments. Although recent studies have demonstrated promising applications of biased estimators, they typically focus only on convergence rates while overlooking generalization performance. We propose a novel multi-metric bias concept, quantified using both local loss and local gradient norm, along with a client activation method based on this bias concept. The proposed method prioritizes training on local datasets that better represent the global dataset, leading to faster convergence and improved generalization. Our extensive empirical study demonstrates that carefully injecting bias into client activation accelerates federated optimization, achieving a substantially improved validation accuracy within a given epoch budget. In representative machine learning benchmarks, our method achieves up to \(12.7\%\) higher accuracy than uniform random sampling and \(2.5\%\) higher accuracy than state-of-the-art biased client activation methods.
Funding Information
  • Institute of Information and Communications Technology Planning and Evaluation (IITP), grant funded by the Korea Government (RS-2022-00155915)
  • Artificial Intelligence Convergence Innovation Human Resources Development
  • National Research Foundation of Korea (NRF), grant funded by the Korea government (No. RS-2024-00452914)

This publication has 16 references indexed in Scilit: