Reward Shaping Based Federated Reinforcement Learning

Authors:
Open Access
Publisher website
Google Scholar
Abstract
Federated reinforcement learning aims to promote training efficiency or improve policy quality through information interaction with privacy protection. Existing federated reinforcement learning methods rarely utilize the structure of reinforcement learning algorithms while are limiting to specific scenarios or algorithms. We propose a general federated reinforcement learning framework FRS, which employs reward shaping as the federated information shared among different clients with different tasks to promote each client’s training speed and policy quality. The federated reward shaping is implicitly learned by average state value information of all clients to protect each client’s task privacy as the real trajectory is anonymous. Experiments on the GridWorld environment show that FRS can algorithm-independently improve the policy quality and promote training speed with protecting each client’s privacy.
Funding Information
  • Foundation Item: AI Project of Shanghai Science and Technology Committee (20DZ1100300, 20511101100)
  • National Natural Science Foundation of China (61972155, 61672231, 12071145)
  • Science Foundation of Shanghai (19ZR1414200)
  • National Key Research and Development Program of China (2020AAA0107400, 2018YFB2101300)

This publication has 17 references indexed in Scilit: