Energy-Based Exploration for Reinforcement Learning of Underactuated Mechanical Systems
Open Access
- 4 June 2025
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 13, 98847-98859
Abstract
Reinforcement Learning (RL) for underactuated mechanical systems presents unique challenges due to the limited control inputs and complex dynamics of the system. Efficient exploration poses a significant problem for two reasons: First, these systems must start each episode of learning from an equilibrium position instead of a random position, severely limiting exploration. Second, these systems are often chaotic, which limits model-based exploration. To address these challenges, we introduce Energy-based Exploration for Reinforcement Learning (EBERL), which exploits a model of the system’s energy represented by a Deep Lagrangian Network (DeLaN), coupled to an energy-based controller to ensure goal-directed exploration on top of the exploration principles that are already present in RL. First, we validated our method in simulation on a Cartpole, Furuta pendulum and Pendubot with challenging torque constraints that cannot be solved by traditional methods. Additionally, we experimentally validated our method on the swing-up and stabilization of a Quanser QUBE2 Furuta pendulum.Keywords
Funding Information
- Strategisch Basis Onderzoek (SBO) and CTRLxAI Project from Fonds Wetenschappelijk Onderzoek (FWO) Research Foundations Flanders (S007723N)
This publication has 29 references indexed in Scilit:
- Reinforcement learning to achieve real-time control of triple inverted pendulumEngineering Applications of Artificial Intelligence, 2023
- Reinforcement Learning DDPG–PPO Agent-Based Control System for Rotary Inverted PendulumArabian Journal for Science and Engineering, 2023
- Curriculum Learning: A SurveyInternational Journal of Computer Vision, 2022
- PBCS: Efficient Exploration and Exploitation Using a Synergy Between Reinforcement Learning and Motion Planning2020
- Fast and slow curiosity for high-level exploration in reinforcement learningApplied Intelligence, 2020
- Swing-up and LQR stabilization of a rotary inverted pendulumArtificial Life and Robotics, 2011
- Gaussian Processes for Sample Efficient Reinforcement Learning with RMAX-Like ExplorationLecture Notes in Computer Science, 2010
- Analysis of the energy‐based swing‐up control of the AcrobotInternational Journal of Robust and Nonlinear Control, 2007
- Control of underactuated mechanical systems using switching and saturation2006
- The transition to chaos in a simple mechanical systemInternational Journal of Non-Linear Mechanics, 1989