Current Organizations / Institutes

The organizations/institutes from the scholar's latest article


Doyoon Ju has a total of 7 co-authors. Here are the top co-authors by number of shared publications:


Latest Publications

  • Open Access

    Reinforcement Learning-Based Transition Control of a Mini Rotary Double Inverted Pendulum: A MuJoCo-Based Approach

    by Sungwon Lee, Doyoon Ju, Young Sam Lee

    This paper proposes a MuJoCo-based sim-to-real reinforcement learning approach for transition control of a mini rotary double inverted pendulum (MRDIP). Transition control between multiple equilibrium points of the MRDIP is challenging because of its underactuated, nonlinear, and strongly coupled dynamics. To address this problem, a direct-drive MRDIP system is developed to enable torque-controlled actuation, and a MuJoCo model is constructed using CAD-based physical parameters and experimentally identified dynamic parameters. Based on the identified model, transition control policies for multiple equilibrium points are trained using the Truncated Quantile Critics (TQC) algorithm. The trained policies are directly implemented on the physical MRDIP and evaluated through successive equilibrium-point transitions without resetting the system between transitions. Experimental results show that the policies trained in MuJoCo successfully perform the intended transitions on the physical system. These results demonstrate the feasibility of the proposed MuJoCo-based framework for sim-to-real transition control of the MRDIP.

  • Open Access

    MuJoCo-Based Sim-to-Real Reinforcement Learning for Transition Control of a Triple Inverted Pendulum

    by Jihun Koo, Sungwon Lee, Doyoon Ju, Young Sam Lee

    This study presents an experimentally validated MuJoCo-based sim-to-real reinforcement learning framework for transition control among all eight equilibrium points (EPs) of a physical triple inverted pendulum (TIP). A multibody model is constructed from the apparatus properties, and uncertain actuation, damping, and friction parameters are identified by matching measured and simulated trajectories using the covariance matrix adaptation evolution strategy. The resulting effective parameter sets are sampled during training using truncated quantile critics to reduce policy dependence on a single model. The trained policies are transferred to the physical TIP without additional tuning or calibration. In the EP0-to-EP7 experiment, the policy trained without parameter identification failed in all ten trials, whereas the policy trained with the identified parameter sets succeeded in all ten. In the sequential experiment, all eight transitions were completed in each of ten trials, yielding 80 successful transitions with no observed failures while satisfying the cart travel and tangential force limits. These results demonstrate sim-to-real transition control across all eight EPs without state resets or precomputed trajectories.

  • Open Access

    Sim-to-Real Reinforcement Learning for a Rotary Double-Inverted Pendulum Based on a Mathematical Model

    by Doyoon Ju, Jongbeom Lee, Young Sam Lee

    This paper proposes a transition control strategy for a rotary double-inverted pendulum (RDIP) system using a sim-to-real reinforcement learning (RL) controller, built upon mathematical modeling and parameter estimation. High-resolution sensor data are used to estimate key physical parameters, ensuring model fidelity for simulation. The resulting mathematical model serves as the training environment in which the RL agent learns to perform transitions between various initial conditions and target equilibrium configurations. The training process adopts the Truncated Quantile Critics (TQC) algorithm, with a reward function specifically designed to reflect the nonlinear characteristics of the system. The learned policy is directly deployed on physical hardware without additional tuning or calibration, and the TQC-based controller successfully achieves all four equilibrium transitions. Furthermore, the controller exhibits robust recovery properties under external disturbances, demonstrating its effectiveness as a reliable sim-to-real control approach for high-dimensional nonlinear systems.


  • Open Access

    Transition Control of a Double-Inverted Pendulum System Using Sim2Real Reinforcement Learning

    by Taegun Lee, Doyoon Ju, Young Sam Lee

    This study presents a sim2real reinforcement learning-based controller for transition control in a double-inverted pendulum system, addressing the limitations of traditional control methods that rely on precomputed trajectories and lack adaptability to strong external disturbances. By introducing the novel concept of ‘transition control’, this research expands the scope of inverted pendulum studies to tackle the challenging task of navigating between multiple equilibrium points. To overcome the reality gap—a persistent challenge in sim2real transfer—a hardware-centered approach was employed, aligning the physical system’s mechanical design with high-fidelity dynamic equations derived from the Euler–Lagrange equation. This design eliminates the need for software-based corrections, ensuring consistent and robust system performance across simulated and real-world environments. Experimental validation demonstrates the controller’s ability to reliably execute all 12 transition scenarios within the double-inverted pendulum system. Additionally, it exhibits recovery characteristics, enabling the system to stabilize and return to equilibrium point even under severe disturbances.


  • Open Access

    Transition Control of a Rotary Double Inverted Pendulum Using Direct Collocation

    by Doyoon Ju, Taegun Lee, Young Sam Lee

    The rotary double inverted pendulum system is characterized by one stable equilibrium point and three unstable equilibrium points due to its kinematic properties. This paper defines the transition control problem between these equilibrium points to extend the conventional swing-up control problem and proposes an implementation method using a laboratory-developed rotary double inverted pendulum. To minimize energy consumption during the transition process while satisfying the boundary conditions of different equilibrium points, a two-point boundary value optimal control problem is formulated. The feedforward trajectory required for feedforward control is computed offline by solving this problem. The direct collocation method is employed to convert the constrained continuous optimal control problem into a nonlinear optimization problem. Furthermore, a time-varying linear–quadratic (LQ) controller is utilized as a feedback controller to accurately track the generated feedforward trajectory during real-time control, compensating for uncertainties in the feedforward control process. The proposed transition control strategy is experimentally implemented, and its effectiveness and practicality are validated through the successful tracking of 12 transition trajectories.


  • Sim-to-Real Reinforcement Learning Techniques for Double Inverted Pendulum Control with Recovery Property

    by Taegun Lee, Doyoon Ju, Young-Sam Lee

    In recent years with the rapid advancement of artificial intelligence, there has been extensive research to address control problems, which was previously unsolvable with traditional control techniques, using reinforcement learning-based controllers. This paper discusses a challenge in controlling a double inverted pendulum system. With the commonly used 2-DOF control technique, once the swing-up control is performed and a strong disturbance is applied, the system becomes uncontrollable and fails to perform another swing-up. However, the reinforcement learning-based controller proposed in this paper overcomes this limitation using the Sim-to-Real learning technique. To ensure successful application of Sim-to-Real learning, this paper proposes a design method for the real-world system that minimizes the reality gap, a chronic issue with the Sim-to-Real technique. Utilizing these techniques, we introduce a characteristic termed ’recovery property’ denoting the ability to recover from strong disturbances, a feature difficult to achieve with traditional control methods. We design a controller with this characteristic and validate its successful operation in a real-world system.

  • Development Environment of Reinforcement Learning-based Controllers for Real-world Physical Systems Using LW-RCP

    by Taegun Lee, Doyoon Ju, Young-Sam Lee

    In recent years, reinforcement learning (RL)-based controller design methods have emerged as a powerful alternatives to traditional methods, providing a novel paradigm that overcomes limitations associated with the need for accurate model information. In this paper, we propose a development environment for RL-based controllers in real-world systems by integrating MATLAB/Simulink, Python, and the LW-RCP (Light-weight Rapid Control Prototyping) system developed by the authors’ laboratory. The proposed development environment utilizes LW-RCP’s library block in a Simulink-based RL controller model, enabling real-time experiments on real-world systems, and stores state information data in MATLAB’s workspace. Python obtains this data through the Python API after each episode and uses it to iteratively enhance the RL agent’s policy by using RL algorithms. Updated parameter values for the agent’s policy neural network are then sent back to MATLAB