Phased Learning with Hierarchical Reinforcement Learning in Nonholonomic Motion Control
Authors:
- 1 January 2006
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 4557-4562
Abstract
In this paper, a hierarchical reinforcement learning algorithm for controlling nonholonomic systems is proposed. When applying reinforcement learning to the nonholonomic systems, acquiring adequate policies is difficult because of an increase of learning steps and a convergence of local optimal policies. The proposed algorithm is inspired by such human learning behavior. Human can learn to control such systems sufficiently even if they initially have little knowledge about the system's dynamics and the way to control. This human capability is suggested to be caused by their exploration strategies for acquiring the adequate policies. The key element of the proposed algorithm is a shaping function defined on a novel position-direction space. The shaping function is autonomously constructed once the goal is reached and constrains the exploration area to optimize the policy. The efficiency of the proposed shaping function was demonstrated by using a nonholonomic control problem of positioning the 2-link planer underactuated manipulatorKeywords
This publication has 15 references indexed in Scilit:
- Analysis of Human Learning Process on Manual Control of Complex Systems2006
- Autonomous shaping2006
- Application of reinforcement learning to balancing of Acrobot2003
- Free-joint manipulators: motion control under second-order nonholonomic constraints2002
- Control of manipulators with free-joints via the averaging method2002
- Towards automatic shaping in robot navigation2002
- Reinforcement Learning in Continuous Time and SpaceNeural Computation, 2000
- Skill modeling through symbolic reconstruction of operator's trajectoriesIEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans, 2000
- Time‐scaling control of an underactuated manipulatorJournal of Robotic Systems, 1998
- Neuronlike adaptive elements that can solve difficult learning control problemsIEEE Transactions on Systems, Man, and Cybernetics, 1983