Abstract
Deep reinforcement learning (DRL) has become increasingly popular for solving sequential decision-making problems in control applications, particularly for multi-physical systems such as vehicle and robotic motion control. The application of DRL to control applications is especially appealing, as control engineering is strongly conducted with the utilization of simulation models. Leveraging these preexisting simulation models for DRL agent training provides a promising and effective strategy. Although DRL has gained popularity in controlling complex multi-physical systems, existing DRL frameworks still lack support for the entire training process of DRL agents utilizing preexisting simulation models. In this work, we present a structured workflow accompanied by its implementation, which utilizes the Functional Mock-up Interface (FMI) to leverage the use of preexisting simulation models for DRL training. We introduce the concept of scenarios as a set of environment parameters and formalize the concept of evaluation metrics. In addition, we extended the scenario concept to uncertain scenarios, to account for environmental uncertainties during training. Our proposed workflow is demonstrated on a path following control problem in automated driving, where a residual policy is learned to enhance the performance of a baseline controller. Using our workflow and toolchain implementation, we successfully trained an agent that achieved significant reductions in tracking errors, with maximum lateral displacement errors reduced by up to 93.4% and velocity tracking errors decreased by up to 41.3%. Our results demonstrates the effectiveness of our approach in leveraging preexisting simulation models for DRL training, paving the way for more efficient and effective control applications.
Funding Information
  • German Aerospace Center

This publication has 33 references indexed in Scilit: