Enhancing Federated Reinforcement Learning: A Consensus-based Approach for Both Homogeneous and Heterogeneous Agents
Open Access
- 27 September 2025
- journal article
- research article
- Published by Springer Nature in Machine Intelligence Research
- Vol. 22 (5) , 929-940
Abstract
Federated reinforcement learning (FedRL) is an emerging paradigm in data-driven control where a group of decision-making agents cooperate to learn optimal control laws through a distributed reinforcement learning procedure, with the peculiarity of having the constraints of not sharing any process/control data. In the typical FedRL setting, a centralized entity is responsible for orchestrating the distributed training process. To remove this design limitation, this work proposes a solution to enable a fully decentralized approach leveraging on results from consensus theory. The proposed algorithm, named FedRLCon, can then deal with: 1) scenarios with homogeneous agents, which can share their actor and, possibly, the critic networks; 2) scenarios with heterogeneous agents, in which agents may share their critic network only. The proposed algorithms are validated on two scenarios, consisting of a resource management problem in a communication network and a smart grid case study. Our tests show that practically no performance is lost for the decentralization.Keywords
This publication has 23 references indexed in Scilit:
- Distributed Policy Gradient with Heterogeneous Computations for Federated Reinforcement Learning2023
- Privacy-Preserving Intelligent Resource Allocation for Federated Edge Learning in Quantum InternetIEEE Journal of Selected Topics in Signal Processing, 2022
- A Weighted Average Consensus Approach for Decentralized Federated LearningMachine Intelligence Research, 2022
- Federated Reinforcement Learning Acceleration Method for Precise Control of Multiple DevicesIEEE Access, 2021
- Multiagent Federated Reinforcement Learning for Secure Incentive Mechanism in Intelligent Cyber–Physical SystemsIEEE Internet of Things Journal, 2021
- On using reinforcement learning for network slice admission control in 5G: Offline vs. onlineInternational Journal of Communication Systems, 2021
- Residential load and rooftop PV generation: an Australian distribution network datasetInternational Journal of Sustainable Energy, 2015
- Q-learningMachine Learning, 1992
- Simple Statistical Gradient-Following Algorithms for Connectionist Reinforcement LearningMachine Learning, 1992
- On the Theory of the Brownian MotionPhysical Review B, 1930