Adaptive reinforcement learning in box-pushing robots
Authors:
- 1 August 2014
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- Vol. 19 (21618070) , 1182-1187
Abstract
In this paper, an adaptive state aggregation Q-Learning method, with the capability of multi-agent cooperation, was proposed to enhance the efficiency of reinforcement learning (RL) and applied to box-pushing tasks for humanoid robots. First, a decision tree was applied to partition the state space according to temporary differences in reinforcement learning, so that a real valued action domain could be represented by a discrete space. Furthermore, adaptive state Q-Learning, which is the modification of estimating Q-value by tabular or function approximation, was proposed to demonstrate the efficiency of reinforcement learning when a humanoid robot pushing a box was simulated. During the process of a robot pushing a box, because the box moves along with the direction the robot asserts force and the pushing point on the box, the robot needs to learn how to adjust angles, avoid obstacles, keep gravity, and push the box to the target point. From the simulation results, the proposed method shows its learning efficiency outperforms the Q-Learning without using adaptive states.Keywords
This publication has 12 references indexed in Scilit:
- Q-learning in multi-agent cooperation2008
- Reinforcement Learning in Continuous Time and Space: Interference and Not Ill Conditioning Is the Main Problem When Using Distributed Function ApproximatorsIEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics), 2008
- Integrating Temporal Difference Methods and Self-Organizing Neural Networks for Reinforcement Learning With Delayed Evaluative FeedbackIEEE Transactions on Neural Networks, 2008
- Tree-like Function Approximator in Reinforcement Learning2007
- State Space Segmentation for Acquisition of Agent Behavior2006
- Control Double Inverted Pendulum by Reinforcement Learning with Double CMAC Network2006
- Variable Resolution Discretization in Optimal ControlMachine Learning, 2002
- Solving Semi-Markov Decision Problems Using Average Reward Reinforcement LearningManagement Science, 1999
- Reinforcement Learning: An IntroductionIEEE Transactions on Neural Networks, 1998
- Linear Least-Squares algorithms for temporal difference learningMachine Learning, 1996