Hierarchical Intermittent Motor Control With Deterministic Policy Gradient

Authors:
Open Access
Publisher website
Google Scholar
Abstract
It has been evidenced that the neural motor control exploits the hierarchical and intermittent representation. In this study we propose a hierarchical deep reinforcement learning method to learn the continuous control policy across multiple levels, by unifying the neuroscience principle of the minimum transition hypothesis. The control policies in the two levels of the hierarchy operate in different time scales. The high-level controller produces the intermittent actions to set a sequence of goals for the low-level controller, which in turn conducts the basic skills with the modulation of goals. The goal planning and the basic motor skills are trained jointly with the proposed algorithm: hierarchical intermittent deep deterministic policy gradient (HI-DDPG). The performance of the method is validated in two continuous control problems. The results show that the method successfully learns to temporally decompose compound tasks into sequences of basic motions with sparse transitions, and outperforms the previous deep reinforcement learning methods that lack a hierarchical continuous representation.
Funding Information
  • National Natural Science Foundation of China (91748122)
  • Science and Technology Commission of Shanghai Municipality (17JC1400603)
  • Natural Science Foundation of Shanghai Municipality (18ZR1442700)

This publication has 33 references indexed in Scilit: