A Heuristic Reinforcement Learning Based on State Backtracking Method
Authors:
- Min FangMFMin Fang
- Hao LiHLHao Li
- Xiaosong ZhangXZXiaosong Zhang
- 1 December 2012
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- Vol. 1, 673-678
Abstract
Since learning action selection strategy is time-consuming due to the reinforcement learning algorithm, a heuristic reinforcement learning algorithm is presented based on the state backtracking reinforcement learning to improve the action selection strategy of the reinforcement learning. The selection strategies of repeated the action are analyzed and compared by state backtracking. A cost function is defined to denote the importance of repetitive actions. A novel heuristic function is given by combing the action-reward with the cost of an action. This algorithm reinforces the important of an action by heuristic function to speed learning and reduces unnecessary explorations by the cost function, so as to steadily improve the learning efficiency. The simulation results of two robot games proves that the algorithm can effectively enhancement the learning rate of Q-learning based on the state backtracking heuristic reinforcement learning method.Keywords
This publication has 4 references indexed in Scilit:
- A Comprehensive Survey of Multiagent Reinforcement LearningIEEE Transactions on Systems, Man and Cybernetics, Part C (Applications and Reviews), 2008
- Automatic shaping and decomposition of reward functions2007
- Accelerating autonomous learning by using heuristic selection of actionsJournal of Heuristics, 2007
- Recent Advances in Hierarchical Reinforcement LearningDiscrete Event Dynamic Systems, 2003