Cost-Effective Federated Learning in Mobile Edge Networks
Top Cited Papers
- 18 October 2021
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Journal on Selected Areas in Communications
- Vol. 39 (12) , 3606-3621
Abstract
Federated learning (FL) is a distributed learning paradigm that enables a large number of mobile devices to collaboratively learn a model under the coordination of a central server without sharing their raw data. Despite its practical efficiency and effectiveness, the iterative on-device learning process (e.g., local computations and global communications with the server) incurs a considerable cost in terms of learning time and energy consumption, which depends crucially on the number of selected clients and the number of local iterations in each training round. In this paper, we analyze how to design adaptive FL in mobile edge networks that optimally chooses these essential control variables to minimize the total cost while ensuring convergence. We establish the analytical relationship between the total cost and the control variables with the convergence upper bound. To efficiently solve the cost minimization problem, we develop a low-cost sampling-based algorithm to learn the convergence related unknown parameters. We derive important solution properties that effectively identify the design principles for different optimization metrics. Practically, we evaluate our theoretical results both in a simulated environment and on a hardware prototype. Experimental evidence verifies our derived properties and demonstrates that our proposed solution achieves near-optimal performance for different optimization metrics for various datasets and heterogeneous system and statistical settings.Keywords
All Related Versions
Funding Information
- Artificial Intelligence and Robotics for Society (AIRS)-Yale Joint Postdoctoral Fellowship
- Shenzhen Science and Technology Program (JCYJ20210324120011032)
- Shenzhen Institute of Artificial Intelligence and Robotics for Society
- Presidential Fund from the Chinese University of Hong Kong, Shenzhen
- AI Institute for Edge Computing Leveraging Next Generation Networks (NSF CNS-2112562, ONR N00014-19-1-2566)
This publication has 26 references indexed in Scilit:
- Scheduling Policies for Federated Learning in Wireless NetworksIEEE Transactions on Communications, 2019
- Parallel Restarted SGD with Faster Convergence and Less Communication: Demystifying Why Model Averaging Works for Deep LearningProceedings of the AAAI Conference on Artificial Intelligence, 2019
- Efficient Deep Learning Hyperparameter Tuning Using Cloud Infrastructure: Intelligent Distributed Hyperparameter Tuning with Bayesian Optimization in the Cloud2019
- Client Selection for Federated Learning with Heterogeneous Resources in Mobile Edge2019
- Federated Learning over Wireless Networks: Optimization Model Design and Analysis2019
- Adaptive Federated Learning in Resource Constrained Edge Computing SystemsIEEE Journal on Selected Areas in Communications, 2019
- A Survey on Mobile Edge Computing: The Communication PerspectiveIEEE Communications Surveys & Tutorials, 2017
- Fog and IoT: An Overview of Research OpportunitiesIEEE Internet of Things Journal, 2016
- Understanding Machine Learning2014
- Biconvex sets and optimization with biconvex functions: a survey and extensionsMathematical Methods of Operations Research, 2007