Yun Hua

In spite of the success of existing meta reinforcement learning methods, they still have difficulty in learning a meta policy effectively for RL problems with sparse reward. In this respect, we develop a novel meta reinforcement learning framework called Hyper-Meta RL(HMRL), for sparse reward RL problems. It is consisted with three modules including the cross-environment meta state embedding module which constructs a common meta state space to adapt to different environments; the meta state based environment-specific meta reward shaping which effectively extends the original sparse reward trajectory by cross-environmental knowledge complementarity and as a consequence the meta policy achieves better generalization and efficiency with the shaped meta reward. Experiments with sparse-reward environments show the superiority of HMRL on both transferability and policy learning efficiency.

Federated reinforcement learning aims to promote training efficiency or improve policy quality through information interaction with privacy protection. Existing federated reinforcement learning methods rarely utilize the structure of reinforcement learning algorithms while are limiting to specific scenarios or algorithms. We propose a general federated reinforcement learning framework FRS, which employs reward shaping as the federated information shared among different clients with different tasks to promote each client’s training speed and policy quality. The federated reward shaping is implicitly learned by average state value information of all clients to protect each client’s task privacy as the real trajectory is anonymous. Experiments on the GridWorld environment show that FRS can algorithm-independently improve the policy quality and promote training speed with protecting each client’s privacy.