Dongjae Kim
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Previous Organizations / Institutes
The organizations / institutes from the scholar's other articles in the past ten years
Research Subjects
Dongjae Kim has a total of 23 co-authors. Here are the top co-authors by number of shared publications:
- Sang Wan Lee
Department of Brain and Cognitive Sciences, Korea Advanced Institute of Science and Technology (KAIST); Kim Jaechul Graduate School of AI, KAIST; KAIST Center for Neuroscience-inspired Artificial Intelligence
5 shared publications - John P. O'DOherty
Division of Humanities & Social Sciences, California Institute of Technology, Pasadena, CA, USA
2 shared publications Sang Wan Lee
2 shared publications- Wei Ji Ma
New York University
2 shared publications Charles Weston
School of Mechanical Engineering, University of Leeds, Leeds, United Kingdom
1 shared publication- Dohyun Kim
Department of Psychiatry, Dankook University, Cheonan-si, South Korea
1 shared publication
Latest Publications
- Open Access
- 14 July 2026
- Published by Springer Nature in Communications Psychology
- Vol. 4 (1)
Decision reversals in sequential decision-making
People often wish they could reverse previous decisions. However, few sequential decision-making tasks used in the literature allow participants to undo their decisions. As a result, it remains an open question when, how, and why people reverse their decisions. We designed a sequential decision-making task in which the participant connected cities on a map using a limited road budget, either without or with the option to undo their decisions. We found that when decisions were reversible, participants spent less time on their first action, progressively improved on their solutions through decision reversals, and ultimately achieved better final performance. Furthermore, decision reversals predominantly corrected errors, and did so with greater precision when errors were larger. Finally, we found evidence that decision reversals were predicted by factors that may be associated with a subjective prior probability of an error. Taken together, our results are consistent with a view in which uncertainty about being on the right track - both retrospective and prospective - drives decision reversals. Our work opens the door to neurocognitive, computational and translational studies of decision reversals.
- 14 July 2026
- Published by Springer Nature in Communications Psychology
- Vol. 4 (1)
Open AccessDecision reversals in sequential decision-making
People often wish they could reverse previous decisions. However, few sequential decision-making tasks used in the literature allow participants to undo their decisions. As a result, it remains an open question when, how, and why people reverse their decisions. We designed a sequential decision-making task in which the participant connected cities on a map using a limited road budget, either without or with the option to undo their decisions. We found that when decisions were reversible, participants spent less time on their first action, progressively improved on their solutions through decision reversals, and ultimately achieved better final performance. Furthermore, decision reversals predominantly corrected errors, and did so with greater precision when errors were larger. Finally, we found evidence that decision reversals were predicted by factors that may be associated with a subjective prior probability of an error. Taken together, our results are consistent with a view in which uncertainty about being on the right track - both retrospective and prospective - drives decision reversals. Our work opens the door to neurocognitive, computational and translational studies of decision reversals.
- Open Access
- 28 October 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 13, 186168-186180
Noise-Robust and Interpretable EEG Classification of Hyperactive Disorders Using Graph Convolutional Network
The analysis of EEG data has been widely utilized due to its non-invasive nature. However, the typically noisy characteristics of EEG signals necessitate artifact removal processes. These processes can involve excessive computation or additional devices, significantly compromising the high usability inherent to non-invasive techniques. Although deep learning was expected to overcome these limitations, it still requires artifact removal and thus becomes highly inefficient. Moreover, it suffers from a lack of interpretability due to its black-box nature. To address these challenges, we propose and validate a framework utilizing Graph Convolutional Network (GCN) and PGExplainer in this study. Specifically, to demonstrate the noise-robustness of our model, we applied it to classify hyperactive disorders such as Tic disorder and Attention deficit hyperactivity disorder (ADHD), which generate excessive noise due to movement. Using EEG data from patients with tics and ADHD, we demonstrated that our model predicts at least 20% higher accuracy than any other neural network models, while the analysis with PGExplainer showed that the same channels were identified as important whether or not noise was removed, indicating noise-robust interpretability of the model. Additionally, this method allowed us to increase computational efficiency, thus identifying a maximally efficient model, i.e., a model that balances accuracy with minimal complexity. The proposed framework is not limited to our case of hyperactive disorder, but is expandable to more general cases. This work demonstrates that GCN-based classification can provide a noise-robust and interpretable solution for EEG analysis.
- 28 October 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 13, 186168-186180
Open AccessNoise-Robust and Interpretable EEG Classification of Hyperactive Disorders Using Graph Convolutional Network
The analysis of EEG data has been widely utilized due to its non-invasive nature. However, the typically noisy characteristics of EEG signals necessitate artifact removal processes. These processes can involve excessive computation or additional devices, significantly compromising the high usability inherent to non-invasive techniques. Although deep learning was expected to overcome these limitations, it still requires artifact removal and thus becomes highly inefficient. Moreover, it suffers from a lack of interpretability due to its black-box nature. To address these challenges, we propose and validate a framework utilizing Graph Convolutional Network (GCN) and PGExplainer in this study. Specifically, to demonstrate the noise-robustness of our model, we applied it to classify hyperactive disorders such as Tic disorder and Attention deficit hyperactivity disorder (ADHD), which generate excessive noise due to movement. Using EEG data from patients with tics and ADHD, we demonstrated that our model predicts at least 20% higher accuracy than any other neural network models, while the analysis with PGExplainer showed that the same channels were identified as important whether or not noise was removed, indicating noise-robust interpretability of the model. Additionally, this method allowed us to increase computational efficiency, thus identifying a maximally efficient model, i.e., a model that balances accuracy with minimal complexity. The proposed framework is not limited to our case of hyperactive disorder, but is expandable to more general cases. This work demonstrates that GCN-based classification can provide a noise-robust and interpretable solution for EEG analysis.
- Open Access
- 21 October 2025
- Published by IOS Press
- p. 281-288
An Arbitration Control for an Ensemble of Diversified DQN Variants in Continual Reinforcement Learning
Deep reinforcement learning (RL) models, despite their efficiency in learning an optimal policy in static environments, easily loses previously learned knowledge (i.e., catastrophic forgetting). It leads RL models to poor performance in continual reinforcement learning (CRL) scenarios. To address this, we present an arbitration control mechanism over an ensemble of RL agents. It is motivated by and closely aligned with how humans make decisions in a CRL context using an arbitration control of multiple RL agents in parallel as observed in the prefrontal cortex. We integrated two key ideas into our model: (1) an ensemble of RLs (i.e., DQN variants) explicitly trained to have diverse value functions and (2) an arbitration control that prioritizes agents with higher reliability (i.e., less error) in recent trials. We propose a framework for CRL, an Arbitration Control for an Ensemble of Diversified DQN variants (ACED-DQN). We demonstrate significant performance improvements in both static and continual environments, supported by empirical evidence showing the effectiveness of arbitration control over diversified DQNs during training. In this work, we introduced a framework that enables RL agents to continuously learn, with inspiration from the human brain.
- 21 October 2025
- Published by IOS Press
- p. 281-288
Open AccessAn Arbitration Control for an Ensemble of Diversified DQN Variants in Continual Reinforcement Learning
Deep reinforcement learning (RL) models, despite their efficiency in learning an optimal policy in static environments, easily loses previously learned knowledge (i.e., catastrophic forgetting). It leads RL models to poor performance in continual reinforcement learning (CRL) scenarios. To address this, we present an arbitration control mechanism over an ensemble of RL agents. It is motivated by and closely aligned with how humans make decisions in a CRL context using an arbitration control of multiple RL agents in parallel as observed in the prefrontal cortex. We integrated two key ideas into our model: (1) an ensemble of RLs (i.e., DQN variants) explicitly trained to have diverse value functions and (2) an arbitration control that prioritizes agents with higher reliability (i.e., less error) in recent trials. We propose a framework for CRL, an Arbitration Control for an Ensemble of Diversified DQN variants (ACED-DQN). We demonstrate significant performance improvements in both static and continual environments, supported by empirical evidence showing the effectiveness of arbitration control over diversified DQNs during training. In this work, we introduced a framework that enables RL agents to continuously learn, with inspiration from the human brain.
- Open Access
An Arbitration Control for an Ensemble of Diversified DQN variants in Continual Reinforcement Learning
Deep reinforcement learning (RL) models, despite their efficiency in learning an optimal policy in static environments, easily loses previously learned knowledge (i.e., catastrophic forgetting). It leads RL models to poor performance in continual reinforcement learning (CRL) scenarios. To address this, we present an arbitration control mechanism over an ensemble of RL agents. It is motivated by and closely aligned with how humans make decisions in a CRL context using an arbitration control of multiple RL agents in parallel as observed in the prefrontal cortex. We integrated two key ideas into our model: (1) an ensemble of RLs (i.e., DQN variants) explicitly trained to have diverse value functions and (2) an arbitration control that prioritizes agents with higher reliability (i.e., less error) in recent trials. We propose a framework for CRL, an Arbitration Control for an Ensemble of Diversified DQN variants (ACED-DQN). We demonstrate significant performance improvements in both static and continual environments, supported by empirical evidence showing the effectiveness of arbitration control over diversified DQNs during training. In this work, we introduced a framework that enables RL agents to continuously learn, with inspiration from the human brain.
Open AccessAn Arbitration Control for an Ensemble of Diversified DQN variants in Continual Reinforcement Learning
Deep reinforcement learning (RL) models, despite their efficiency in learning an optimal policy in static environments, easily loses previously learned knowledge (i.e., catastrophic forgetting). It leads RL models to poor performance in continual reinforcement learning (CRL) scenarios. To address this, we present an arbitration control mechanism over an ensemble of RL agents. It is motivated by and closely aligned with how humans make decisions in a CRL context using an arbitration control of multiple RL agents in parallel as observed in the prefrontal cortex. We integrated two key ideas into our model: (1) an ensemble of RLs (i.e., DQN variants) explicitly trained to have diverse value functions and (2) an arbitration control that prioritizes agents with higher reliability (i.e., less error) in recent trials. We propose a framework for CRL, an Arbitration Control for an Ensemble of Diversified DQN variants (ACED-DQN). We demonstrate significant performance improvements in both static and continual environments, supported by empirical evidence showing the effectiveness of arbitration control over diversified DQNs during training. In this work, we introduced a framework that enables RL agents to continuously learn, with inspiration from the human brain.
- Open Access
- 8 August 2025
- Published by MDPI AG in Mathematics
- Vol. 13 (16) , 2542
Uncertainty-Aware Continual Reinforcement Learning via PPO with Graph Representation Learning
Continual reinforcement learning (CRL) agents face significant challenges when encountering distributional shifts. This paper formalizes these shifts into two key scenarios, namely virtual drift (domain switches), where object semantics change (e.g., walls becoming lava), and concept drift (task switches), where the environment’s structure is reconfigured (e.g., moving from object navigation to a door key puzzle). This paper demonstrates that while conventional convolutional neural networks (CNNs) struggle to preserve relational knowledge during these transitions, graph convolutional networks (GCNs) can inherently mitigate catastrophic forgetting by encoding object interactions through explicit topological reasoning. A unified framework is proposed that integrates GCN-based state representation learning with a proximal policy optimization (PPO) agent. The GCN’s message-passing mechanism preserves invariant relational structures, which diminishes performance degradation during abrupt domain switches. Experiments conducted in procedurally generated MiniGrid environments show that the method significantly reduces catastrophic forgetting in domain switch scenarios. While showing comparable mean performance in task switch scenarios, our method demonstrates substantially lower performance variance (Levene’s test, ), indicating superior learning stability compared to CNN-based methods. By bridging graph representation learning with robust policy optimization in CRL, this research advances the stability of decision-making in dynamic environments and establishes GCNs as a principled alternative to CNNs for applications requiring stable, continual learning.- 8 August 2025
- Published by MDPI AG in Mathematics
- Vol. 13 (16) , 2542
Open AccessUncertainty-Aware Continual Reinforcement Learning via PPO with Graph Representation Learning
Continual reinforcement learning (CRL) agents face significant challenges when encountering distributional shifts. This paper formalizes these shifts into two key scenarios, namely virtual drift (domain switches), where object semantics change (e.g., walls becoming lava), and concept drift (task switches), where the environment’s structure is reconfigured (e.g., moving from object navigation to a door key puzzle). This paper demonstrates that while conventional convolutional neural networks (CNNs) struggle to preserve relational knowledge during these transitions, graph convolutional networks (GCNs) can inherently mitigate catastrophic forgetting by encoding object interactions through explicit topological reasoning. A unified framework is proposed that integrates GCN-based state representation learning with a proximal policy optimization (PPO) agent. The GCN’s message-passing mechanism preserves invariant relational structures, which diminishes performance degradation during abrupt domain switches. Experiments conducted in procedurally generated MiniGrid environments show that the method significantly reduces catastrophic forgetting in domain switch scenarios. While showing comparable mean performance in task switch scenarios, our method demonstrates substantially lower performance variance (Levene’s test, ), indicating superior learning stability compared to CNN-based methods. By bridging graph representation learning with robust policy optimization in CRL, this research advances the stability of decision-making in dynamic environments and establishes GCNs as a principled alternative to CNNs for applications requiring stable, continual learning. - Open Access
- 28 June 2023
- Published by Springer Nature in Research Square
Long short-term prediction guides human metacognitive reinforcement learning
Long-term prediction ensures stable but slow learning. Short-term prediction is fast but deemed to be risky in context-changing environments. To understand how humans resolve this conflict, we fitted nine biologically plausible reinforcement learning models to 82 human behavior data using a human-model policy matching method. One meta-RL model replicates subjects' context-dependent behavior, passing underfitting and overfitting tests. The same model survived a generalizability test simulating various Markov decision tasks and an adaptation test in which task structure, reward, and uncertainty change unexpectedly. Critically, the model’s ability is attributed to long- and short-term predictions about state transition and reward. Quantitative analyses revealed that conventional instant prediction error and its underlying long-term baseline are sufficient for context embeddings. These findings generate a long short-term prediction hypothesis: humans translate short-term into long-term predictions for context generalization while self-evaluating their long-term predictions to adjust short-term predictions. This creates a virtuous cycle of metacognitive generalization-adaptation.
- 28 June 2023
- Published by Springer Nature in Research Square
Open AccessLong short-term prediction guides human metacognitive reinforcement learning
Long-term prediction ensures stable but slow learning. Short-term prediction is fast but deemed to be risky in context-changing environments. To understand how humans resolve this conflict, we fitted nine biologically plausible reinforcement learning models to 82 human behavior data using a human-model policy matching method. One meta-RL model replicates subjects' context-dependent behavior, passing underfitting and overfitting tests. The same model survived a generalizability test simulating various Markov decision tasks and an adaptation test in which task structure, reward, and uncertainty change unexpectedly. Critically, the model’s ability is attributed to long- and short-term predictions about state transition and reward. Quantitative analyses revealed that conventional instant prediction error and its underlying long-term baseline are sufficient for context embeddings. These findings generate a long short-term prediction hypothesis: humans translate short-term into long-term predictions for context generalization while self-evaluating their long-term predictions to adjust short-term predictions. This creates a virtuous cycle of metacognitive generalization-adaptation.
- Open Access
Reward prediction error neurons implement an efficient code for reward
We use efficient coding principles borrowed from sensory neuroscience to derive the optimal population of neurons to encode rewards from a probability distribution. We find that the response properties of dopaminergic reward prediction error neurons in a rodent and a primate data set are similar to those of the efficient code in many ways: the neurons have a broad distribution of midpoints covering the reward distribution; neurons with higher thresholds have higher gains, more convex tuning functions, and lower slopes; moreover, their slope is higher when the reward distribution is narrower. Furthermore, we derive learning rules that converge to this efficient code; the learning rule for the position of the neuron on the reward axis closely resembles the learning rule of distributional reinforcement learning. Thus, reward prediction error neuron responses may be optimized to broadcast an efficient reward signal, forming a connection between efficient coding and reinforcement learning, two of the most successful theories in computational neuroscience.
Open AccessReward prediction error neurons implement an efficient code for reward
We use efficient coding principles borrowed from sensory neuroscience to derive the optimal population of neurons to encode rewards from a probability distribution. We find that the response properties of dopaminergic reward prediction error neurons in a rodent and a primate data set are similar to those of the efficient code in many ways: the neurons have a broad distribution of midpoints covering the reward distribution; neurons with higher thresholds have higher gains, more convex tuning functions, and lower slopes; moreover, their slope is higher when the reward distribution is narrower. Furthermore, we derive learning rules that converge to this efficient code; the learning rule for the position of the neuron on the reward axis closely resembles the learning rule of distributional reinforcement learning. Thus, reward prediction error neuron responses may be optimized to broadcast an efficient reward signal, forming a connection between efficient coding and reinforcement learning, two of the most successful theories in computational neuroscience.
- Open Access
Prefrontal solution to the bias-variance tradeoff during reinforcement learning
The goal of learning is to maximize future rewards by minimizing prediction errors. Evidence have shown that the brain achieves this by combining model-based and model-free learning. However, the prediction error minimization is challenged by a bias-variance tradeoff, which imposes constraints on each strategy’s performance. We provide new theoretical insight into how this tradeoff can be resolved through the adaptive control of model-based and model-free learning. The theory predicts the baseline correction for prediction error reduces the lower bound of the bias–variance error by factoring out irreducible noise. Using a Markov decision task with context changes, we showed behavioral evidence of adaptive control. Model-based behavioral analyses show that the prediction error baseline signals context changes to improve adaptability. Critically, the neural results support this view, demonstrating multiplexed representations of prediction error baseline within the ventrolateral and ventromedial prefrontal cortex, key brain regions known to guide model-based and model-free learning. One sentence summary A theoretical, behavioral, computational, and neural account of how the brain resolves the bias-variance tradeoff during reinforcement learning is described.
Open AccessPrefrontal solution to the bias-variance tradeoff during reinforcement learning
The goal of learning is to maximize future rewards by minimizing prediction errors. Evidence have shown that the brain achieves this by combining model-based and model-free learning. However, the prediction error minimization is challenged by a bias-variance tradeoff, which imposes constraints on each strategy’s performance. We provide new theoretical insight into how this tradeoff can be resolved through the adaptive control of model-based and model-free learning. The theory predicts the baseline correction for prediction error reduces the lower bound of the bias–variance error by factoring out irreducible noise. Using a Markov decision task with context changes, we showed behavioral evidence of adaptive control. Model-based behavioral analyses show that the prediction error baseline signals context changes to improve adaptability. Critically, the neural results support this view, demonstrating multiplexed representations of prediction error baseline within the ventrolateral and ventromedial prefrontal cortex, key brain regions known to guide model-based and model-free learning. One sentence summary A theoretical, behavioral, computational, and neural account of how the brain resolves the bias-variance tradeoff during reinforcement learning is described.
- Open Access
- 16 December 2019
- Published by Springer Nature in Nature Communications
- Vol. 10 (1) , 1-14
Task complexity interacts with state-space uncertainty in the arbitration between model-based and model-free learning
It has previously been shown that the relative reliability of model-based and model-free reinforcement-learning (RL) systems plays a role in the allocation of behavioral control between them. However, the role of task complexity in the arbitration between these two strategies remains largely unknown. Here, using a combination of novel task design, computational modelling, and model-based fMRI analysis, we examined the role of task complexity alongside state-space uncertainty in the arbitration process. Participants tended to increase model-based RL control in response to increasing task complexity. However, they resorted to model-free RL when both uncertainty and task complexity were high, suggesting that these two variables interact during the arbitration process. Computational fMRI revealed that task complexity interacts with neural representations of the reliability of the two systems in the inferior prefrontal cortex.
- 16 December 2019
- Published by Springer Nature in Nature Communications
- Vol. 10 (1) , 1-14
Open AccessTask complexity interacts with state-space uncertainty in the arbitration between model-based and model-free learning
It has previously been shown that the relative reliability of model-based and model-free reinforcement-learning (RL) systems plays a role in the allocation of behavioral control between them. However, the role of task complexity in the arbitration between these two strategies remains largely unknown. Here, using a combination of novel task design, computational modelling, and model-based fMRI analysis, we examined the role of task complexity alongside state-space uncertainty in the arbitration process. Participants tended to increase model-based RL control in response to increasing task complexity. However, they resorted to model-free RL when both uncertainty and task complexity were high, suggesting that these two variables interact during the arbitration process. Computational fMRI revealed that task complexity interacts with neural representations of the reliability of the two systems in the inferior prefrontal cortex.
- 1 October 2018
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 990-994
Hierarchical Control Architecture Regulating Competition between Model-Based and Context-Dependent Model-Free Reinforcement Learning Strategies
Recent evidence in neuroscience and psychology suggests that a single reinforcement learning (RL) algorithm only accounts for less than 60% of the variance of human choice behavior in an uncertain and dynamic environment, where the amount of uncertainty in state-action-state transitions drift over time. The prediction performance further decreases when the size of the state space increases. We proposed a hierarchical context-dependent RL control framework that dynamically exerted control weights on model-based (MB) and multiple model-free (MF) RL strategies associated with different task goals. To properly assess the validity of the proposed method, we considered a two-stage Markov decision task (MDT) in which the three different types of context changed over time. We trained 57 different RL control models on a Caltech MDT data set; then, we assessed their prediction performance using a Bayesian model comparison. This large-scale computer simulation analysis revealed that the model providing the most accurate prediction was the version that implemented the competition between the MB and multiple goal-dependent MF RL strategies. The present study demonstrates the applicability of the goal-driven RL control to a variety of real-world human-robot interaction scenarios.
- 1 October 2018
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 990-994
Hierarchical Control Architecture Regulating Competition between Model-Based and Context-Dependent Model-Free Reinforcement Learning Strategies
Recent evidence in neuroscience and psychology suggests that a single reinforcement learning (RL) algorithm only accounts for less than 60% of the variance of human choice behavior in an uncertain and dynamic environment, where the amount of uncertainty in state-action-state transitions drift over time. The prediction performance further decreases when the size of the state space increases. We proposed a hierarchical context-dependent RL control framework that dynamically exerted control weights on model-based (MB) and multiple model-free (MF) RL strategies associated with different task goals. To properly assess the validity of the proposed method, we considered a two-stage Markov decision task (MDT) in which the three different types of context changed over time. We trained 57 different RL control models on a Caltech MDT data set; then, we assessed their prediction performance using a Bayesian model comparison. This large-scale computer simulation analysis revealed that the model providing the most accurate prediction was the version that implemented the competition between the MB and multiple goal-dependent MF RL strategies. The present study demonstrates the applicability of the goal-driven RL control to a variety of real-world human-robot interaction scenarios.