ASMAFL: Adaptive Staleness-Aware Momentum Asynchronous Federated Learning in Edge Computing
- 2 December 2024
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Mobile Computing
- Vol. 24 (4) , 3390-3406
Abstract
Compared with synchronous federated learning (FL), asynchronous FL (AFL) has attracted more and more attention in edge computing (EC) fields because of its strong adaptability to heterogeneous application scenarios. However, the non-independent and identically distributed (Non-IID) data across devices and the staleness-aware estimation of unreliable wireless connections and limited edge resources make it much more difficult to achieve better AFL-related applications. To handle this problem, we propose an Adaptive Staleness-aware Momentum Accelerated AFL (ASMAFL) algorithm to reduce the resources consumption of heterogeneous wireless communication EC (WCEC) scenarios, as well as decrease the negative impact of Non-IID data for model training. Specifically, we first introduce the staleness-aware parameter and a unified momentum gradient descent (GD) framework to reformulate AFL. Then, we establish global convergence properties of AFL, derive an upper bound on AFL convergence rate, and find that the bound is related to the staleness-aware parameter and Non-IIDness. Next, we formulate the bound into a minimization problem of resource consumption under given model accuracy, and the corresponding staleness-aware parameter of devices will be recomputed after each asynchronous aggregation to eliminate the differences of local models’ contribution to global model aggregation. Finally, extensive experiments are carried out to validate the superiority of ASMAFL in model accuracy, convergence rate, resources consumption, Non-IID issue, etc.Keywords
Funding Information
- National Natural Science Foundation of China (62272069, 62203077, 62202071)
- Natural Science Key Foundation of Chongqing (CSTB2024NSCQ-LZX0129)
- Fundamental Research Funds for the Central Universities (2024CDJXY013, 2023CDJYGRH-ZD06)
- Natural Science Foundation of Chongqing Municipality (CSTB2022NSCQ-MSX1029)
- Special Research Funding for Chongqing Postdoctoral Researchers (2021XM2015)
- Science-Health Joint Medical Scientific Research Project of Chongqing (2023MSXM013)
This publication has 39 references indexed in Scilit:
- AsyFed: Accelerated Federated Learning With Asynchronous Communication MechanismIEEE Internet of Things Journal, 2022
- Tackling System and Statistical Heterogeneity for Federated Learning with Adaptive Client Sampling2022
- Gradient Scheduling With Global Momentum for Asynchronous Federated Learning in Edge EnvironmentIEEE Internet of Things Journal, 2022
- FedSA: A staleness-aware asynchronous Federated Learning algorithm with non-IID dataFuture Generation Computer Systems, 2021
- Federated Learning for Internet of Things: Recent Advances, Taxonomy, and Open ChallengesIEEE Communications Surveys & Tutorials, 2021
- Asynchronous Online Federated Learning for Edge Devices with Non-IID Data2020
- Adaptive Federated Learning in Resource Constrained Edge Computing SystemsIEEE Journal on Selected Areas in Communications, 2019
- The EU General Data Protection Regulation (GDPR)2017
- Global convergence of the Heavy-ball method for convex optimization2015
- Some methods of speeding up the convergence of iteration methodsUSSR Computational Mathematics and Mathematical Physics, 1964