Decoupled Reinforcement Hybrid PPO–Sliding Control for Underactuated Systems: Application to Cart–Pole and Acrobot
Author:
Open Access
Abstract
Underactuated systems, such as the Cart–Pole and Acrobot, pose significant control challenges due to their inherent nonlinearity and limited actuation. Traditional control methods often struggle to achieve stable and optimal performance in these complex scenarios. This paper presents a novel stable reinforcement learning (RL) approach for underactuated systems, integrating advanced exploration–exploitation mechanisms and a refined policy optimization framework to address instability issues in RL-based control. The proposed method is validated through extensive experiments on two benchmark underactuated systems: the Cart–Pole and Acrobot. In the Cart–Pole task, the method achieves long-term balance with high stability, outperforming traditional RL algorithms such as the Proximal Policy Optimization (PPO) in average episode length and robustness to environmental disturbances. For the Acrobot, the approach enables reliable swing-up and near-vertical stabilization but cannot achieve sustained balance control beyond short time intervals due to residual dynamics and control limitations. A key contribution is the development of a hybrid PPO–sliding mode control strategy that enhances learning efficiency and stabilities for underactuated systems.Keywords
This publication has 22 references indexed in Scilit:
- Deep Reinforcement Learning-Based Uncalibrated Visual Servoing Control of Manipulators with FOV ConstraintsApplied Sciences, 2025
- Tikhonov-Tuned Sliding Neural Network Decoupling Control for an Inverted PendulumElectronics, 2023
- Accelerating Fuzzy Actor–Critic Learning via Suboptimal Knowledge for a Multi-Agent Tracking ProblemElectronics, 2023
- Reinforcement Learning for Decentralized Stochastic Control2019
- Human-level control through deep reinforcement learningNature, 2015
- Reinforcement Learning: A SurveyJournal of Artificial Intelligence Research, 1996
- Q-learningMachine Learning, 1992
- Simple Statistical Gradient-Following Algorithms for Connectionist Reinforcement LearningMachine Learning, 1992
- Neuronlike adaptive elements that can solve difficult learning control problemsIEEE Transactions on Systems, Man, and Cybernetics, 1983
- Some Studies in Machine Learning Using the Game of CheckersIBM Journal of Research and Development, 1959