Automated Curriculum Reinforcement Learning in Path-finding Scenarios

Authors:
Publisher website
Google Scholar
Abstract
Despite the broad applications of reinforcement learning in various fields, low training efficiency caused by the sparse reward problem remains crucial. In the context of pathfinding, consider the following question: given a limited number of expert trajectories, how can we find an efficient strategy that allows an agent to be trained in a shorter time achieving similar training performance in complex environments? In this paper, we propose a training strategy based on the results of inverse reinforcement learning, which not only performs well in the same environment but also reduces training time. The idea is to apply Gaussian-based inverse reinforcement learning to analyze expert trajectories, inferring reward functions and clustering. Then the framework of curriculum learning is introduced to shorten the training time and improve the training efficiency under complex tasks and extensive data. The experimental section explores the general rules for the optimal number of intermediate points. Ablation experiments were set up to demonstrate the scientific validity of the middle point selection of the curriculum. Additionally, we compare the time and reward function of our proposed method with baselines under different map sizes, numbers of obstacles, and distances to evaluate the learning performance in various scenarios. Based on the experimental results, our proposed approach outperforms the best baseline in most cases and achieves training performance similar to expert trajectories with less training time, solving the sparse reward problem.
Funding Information
  • Fundamental Research Funds for the Central Universities
  • National Natural Science Foundation of China
  • National Key Research and Development Program of China

This publication has 14 references indexed in Scilit: