Toward Dynamic Resource Allocation and Client Scheduling in Hierarchical Federated Learning: A Two-Phase Deep Reinforcement Learning Approach
- 2 July 2024
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Communications
- Vol. 72 (12) , 7798-7813
Abstract
Federated learning (FL) is a viable technique to train a shared machine learning model without sharing data. Hierarchical FL (HFL) system has yet to be studied regrading its multiple levels of energy, computation, communication, and client scheduling, especially when it comes to clients relying on energy harvesting to power their operations. This paper presents a new two-phase deep deterministic policy gradient (DDPG) framework, referred to as “TP-DDPG”, to balance online the learning delay and model accuracy of an FL process in an energy harvesting-powered HFL system. The key idea is that we divide optimization decisions into two groups, and employ DDPG to learn one group in the first phase, while interpreting the other group as part of the environment to provide rewards for training the DDPG in the second phase. Specifically, the DDPG learns the selection of participating clients, and their CPU configurations and the transmission powers. A new straggler-aware client association and bandwidth allocation (SCABA) algorithm efficiently optimizes the other decisions and evaluates the reward for the DDPG. Experiments demonstrate that with substantially reduced number of learnable parameters, the TP-DDPG can quickly converge to effective polices that can shorten the training time of HFL by 39.4% compared to its benchmarks, when the required test accuracy of HFL is 0.9.Keywords
All Related Versions
Funding Information
- National Key Research and Development Program of China (2022YFB2902002, 2022YFB2902303)
- Innovation Program of Shanghai Municipal Science and Technology Commission (21ZR1422400)
- Program of Hebei Key Laboratory of Advanced Laser Technology and Equipment (HBKL-ALTE202401)
- National High Quality Program (TC220H07D)
- Shanghai Rising-Star Program
This publication has 43 references indexed in Scilit:
- Poisoning Attacks in Federated Learning: A SurveyIEEE Access, 2023
- Two-Phase Deep Reinforcement Learning of Dynamic Resource Allocation and Client Selection for Hierarchical Federated Learning2022
- Communication-efficient hierarchical federated learning for IoT heterogeneous systems with imbalanced dataFuture Generation Computer Systems, 2022
- SHARE: Shaping Data Distribution at Edge for Communication-Efficient Hierarchical Federated Learning2021
- Distributed Machine Learning for Wireless Communication Networks: Techniques, Architectures, and ApplicationsIEEE Communications Surveys & Tutorials, 2021
- Resource-Efficient Federated Learning with Hierarchical Aggregation in Edge Computing2021
- Joint User Scheduling and Resource Allocation for Federated Learning over Wireless Networks2020
- Client-Edge-Cloud Hierarchical Federated Learning2020
- Federated Learning in Mobile Edge Networks: A Comprehensive SurveyIEEE Communications Surveys & Tutorials, 2020
- Client Selection for Federated Learning with Heterogeneous Resources in Mobile Edge2019