train_fmu_gym: A Functional Mock-up Unit-Based Framework to Train Reinforcement Learning Agents for Multi-Physical Systems
Open Access
- 25 November 2025
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 13, 208845-208867
Abstract
Deep reinforcement learning (DRL) has become increasingly popular for solving sequential decision-making problems in control applications, particularly for multi-physical systems such as vehicle and robotic motion control. The application of DRL to control applications is especially appealing, as control engineering is strongly conducted with the utilization of simulation models. Leveraging these preexisting simulation models for DRL agent training provides a promising and effective strategy. Although DRL has gained popularity in controlling complex multi-physical systems, existing DRL frameworks still lack support for the entire training process of DRL agents utilizing preexisting simulation models. In this work, we present a structured workflow accompanied by its implementation, which utilizes the Functional Mock-up Interface (FMI) to leverage the use of preexisting simulation models for DRL training. We introduce the concept of scenarios as a set of environment parameters and formalize the concept of evaluation metrics. In addition, we extended the scenario concept to uncertain scenarios, to account for environmental uncertainties during training. Our proposed workflow is demonstrated on a path following control problem in automated driving, where a residual policy is learned to enhance the performance of a baseline controller. Using our workflow and toolchain implementation, we successfully trained an agent that achieved significant reductions in tracking errors, with maximum lateral displacement errors reduced by up to 93.4% and velocity tracking errors decreased by up to 41.3%. Our results demonstrates the effectiveness of our approach in leveraging preexisting simulation models for DRL training, paving the way for more efficient and effective control applications.Keywords
Funding Information
- German Aerospace Center
This publication has 33 references indexed in Scilit:
- A survey on reinforcement learning in aviation applicationsEngineering Applications of Artificial Intelligence, 2024
- Safety-Driven Deep Reinforcement Learning Framework for Cobots: A Sim2Real Approach2024
- A novel Deep Reinforcement Learning based automated stock trading system using cascaded LSTM networksExpert Systems with Applications, 2023
- Learning-Based Control for Hybrid Battery Management Systems2022
- Array programming with NumPyNature, 2020
- Reinforcement learning for intelligent healthcare applications: A surveyArtificial Intelligence in Medicine, 2020
- Multi-domain Modeling and Simulation2019
- Mastering the game of Go with deep neural networks and tree searchNature, 2016
- Human-level control through deep reinforcement learningNature, 2015
- EnergyPlus: creating a new-generation building energy simulation programEnergy and Buildings, 2001