DLDonghwan Lee

Donghwan Lee

*This is an algorithmically generated author record based on publicly available information.

Current Organizations / Institutes

The organizations/institutes from the scholar's latest article

Previous Organizations / Institutes

The organizations / institutes from the scholar's other articles in the past ten years


Donghwan Lee has a total of 55 co-authors. Here are the top co-authors by number of shared publications:


Latest Publications

  • Open Access

    TD-Learning and Q-Learning: A Survey of Theory, Analysis, and Trends

    by Han-Dong Lim, Hyunjun Na, Taeho Lee, Donghwan Lee

    This paper provides a comprehensive survey of the convergence properties of temporal-difference (TD) learning and Q-learning, which are two fundamental algorithms in reinforcement learning (RL). We systematically categorize the existing literature into the tabular setting and two function approximation regimes: linear and nonlinear (deep neural networks). In the tabular setting, we review the foundational stochastic-approximation theory that ensures asymptotic convergence and discuss recent non-asymptotic results that provide explicit sample-complexity bounds under various coverage and sampling conditions. For linear function approximation, we address the stability challenges inherent in off-policy learning and examine stabilization mechanisms, such as gradient-based methods, regularization, and target networks. Furthermore, we explore the recent theoretical advancements in deep RL, focusing on finite-time analysis within the overparameterized regime. By synthesizing these diverse perspectives, this survey highlights the theoretical evolution from asymptotic stability to non-asymptotic efficiency and identifies the remaining gaps and provides a coherent roadmap for future research toward a unified theoretical understanding of RL dynamics.

  • Open Access

    Regional Output-Feedback Control for T–S Fuzzy Systems Using Lyapunov Functions With Polynomial Dependency on States and Membership Functions

    by Yara Quilles-Marinho, Donghwan Lee, Ricardo C. L. F. Oliveira, Pedro L. D. Peres

    This article addresses the problem of local output-feedback control design for continuous-time Takagi–Sugeno fuzzy systems using the parallel distributed compensation (PDC) approach. The main contribution is the introduction of a novel class of Lyapunov functions with polynomial dependence on both the system states and the membership functions (MFs), providing less conservative stability and synthesis conditions. To compute the PDC gains, a locally convergent iterative algorithm based on linear matrix inequalities (LMIs) is proposed. Unlike many existing methods, the approach eliminates the need for bounds on the time derivatives of MFs by explicitly incorporating their dynamics into the stability conditions. The algorithm operates in two phases: first, optimizing the decay rate of the closed-loop trajectories, andsecond, maximizing the estimated domain of attraction (DOA) under a fixed decay rate. Comparative numerical examples based on benchmark systems demonstrate the superior performance of the proposed method over existing techniques, particularly as the polynomial degree of the Lyapunov function increases.

  • Open Access

    Finite-Time Accuracy of Temporal-Difference Learning Under Schur-Stable Recursions

    by Donghwan Lee, Dowan Kim

    Temporal difference (TD) learning is a cornerstone reinforcement learning (RL) method for policy evaluation, where the goal is to estimate the value function of a Markov decision process under a fixed policy. While a substantial body of work has established its convergence and stability properties, more recent efforts have focused on its statistical efficiency through finite-time error bounds. In this paper, we advance this line of research by developing a new finite-time error analysis for tabular TD learning that directly exploits a discrete-time stochastic linear system representation and leverages Schur stability of the associated matrices. Beyond the specific bounds obtained, the proposed framework provides a reusable template for analyzing TD learning and related RL algorithms, and it offers control-theoretic insights that may guide future developments in finite-sample RL theory.

  • Open Access

    Deep Q-Learning With Gradient Target Tracking

    by Bumgeun Park, Taeho Lee, Donghwan Lee

    This paper introduces Q-learning with gradient target tracking, a novel reinforcement learning framework that provides a learned continuous target update mechanism as an alternative to the conventional hard update paradigm. In the standard deep Q-network (DQN), the target network is a copy of the online network’s weights, held fixed for a number of iterations before being periodically replaced via a hard update. While this stabilizes training by providing consistent targets, it introduces a new challenge: the hard update period must be carefully tuned to achieve optimal performance. To address this issue, we propose two gradient-based target update methods: DQN with asymmetric gradient target tracking (AGT2-DQN) and DQN with symmetric gradient target tracking (SGT2-DQN). These methods replace the conventional hard target updates with continuous and structured updates using gradient descent, which effectively eliminates the need for manual tuning. We provide a theoretical analysis proving the convergence of these methods in tabular settings. Additionally, empirical evaluations demonstrate their advantages over standard DQN baselines, which suggest that gradient-based target updates can serve as an effective alternative to conventional target update mechanisms in Q-learning.

  • Open Access

    High-Precision Temperature Control in Industrial Evaporation Heaters via Hybrid Reinforcement Learning Approach

    by Bumgeun Park, Narim Jeong, Hyukjun Yang, Dong Seok Park, In Je Shin, Jae Won Kim, Donghwan Lee

  • Open Access

    Regional Stabilization and Estimation of Domains of Attraction for Discrete-Time T–S Fuzzy Systems via Fuzzy-modeled Membership Functions

    by Yara Quilles-Marinho, Donghwan Lee, Ricardo C. L. F. Oliveira, Pedro L. D. Peres

    This paper addresses the problem of regional stabilization for discrete-time Takagi-Sugeno fuzzy systems using parallel distributed compensation output-feedback controllers. The goal is to ensure asymptotic stability and to maximize the estimate of the corresponding domain of attraction. The key contribution of this work is the use of a fuzzy modeling approach for the membership functions, enabling a polytopic representation based on the premise variables. Unlike conventional approaches, this method avoids the need for explicit bounds on membership function variations, thereby reducing conservatism in the stabilization conditions. The synthesis procedure is presented as an iterative algorithm based on linear matrix inequalities, consisting of two distinct phases: the first phase stabilizes the system, while the second phase aims to maximize the estimated region of attraction. Numerical examples are provided to demonstrate the advantages of the proposed technique over existing methods.


  • Open Access

    Adaptive Policy Backbone via Shared Network

    by Bumgeun Park, Donghwan Lee

    Reinforcement learning (RL) has achieved impressive results across domains, yet learning an optimal policy typically requires extensive interaction data, limiting practical deployment. A common remedy is to leverage priors, such as pre-collected datasets or reference policies, but their utility degrades under task mismatch between training and deployment. While prior work has sought to address this mismatch, it has largely been restricted to in-distribution settings. To address this challenge, we propose Adaptive Policy Backbone (APB), a meta-transfer RL method that inserts lightweight linear layers before and after a shared backbone, thereby enabling parameter-efficient fine-tuning (PEFT) while preserving prior knowledge during adaptation. Our results show that APB improves sample efficiency over standard RL and adapts to out-of-distribution (OOD) tasks where existing meta-RL baselines typically fail.

  • Open Access

    A Primal-dual Perspective for Distributed TD-learning

    by Han Dong Lim, Donghwan Lee

    The goal of this paper is to investigate distributed temporal difference (TD) learning for a networked multi-agent Markov decision process. The proposed approach is based on distributed optimization algorithms, which can be interpreted as primal-dual ordinary differential equation (ODE) dynamics subject to null-space constraints. Based on the exponential convergence behavior of the primal-dual ODE dynamics subject to null-space constraints, we examine the behavior of the final iterate in various distributed TD-learning scenarios, considering both constant and diminishing step-sizes and incorporating both i.i.d. and Markovian observation models. Unlike existing methods, the proposed algorithm does not require the assumption that the underlying communication network structure is characterized by a doubly stochastic matrix.

  • Open Access

    Robust Deterministic Policy Gradient for Disturbance Attenuation and Its Application to Quadrotor Control

    by Taeho Lee, Donghwan Lee

    Practical control systems pose significant challenges in identifying optimal control policies due to uncertainties in the system model and external disturbances. While \(H_\infty\) control techniques are commonly used to design robust controllers that mitigate the effects of disturbances, these methods often require complex and computationally intensive calculations. To address this issue, this paper proposes a reinforcement learning algorithm called Robust Deterministic Policy Gradient (RDPG), which formulates the $H_\infty$ control problem as a two-player zero-sum dynamic game. In this formulation, one player (the user) aims to minimize the cost, while the other player (the adversary) seeks to maximize it. We then employ deterministic policy gradient (DPG) and its deep reinforcement learning counterpart to train a robust control policy with effective disturbance attenuation. In particular, for practical implementation, we introduce an algorithm called robust deep deterministic policy gradient (RDDPG), which employs a deep neural network architecture and integrates techniques from the twin-delayed deep deterministic policy gradient (TD3) to enhance stability and learning efficiency.To evaluate the proposed algorithm, we implement it on an unmanned aerial vehicle (UAV) tasked with following a predefined path in a disturbance-prone environment. The experimental results demonstrate that the proposed method outperforms other control approaches in terms of robustness against disturbances, enabling precise real-time tracking of moving targets even under severe disturbance conditions.