Hyun-Rok Lee
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Hyun-Rok Lee has a total of 16 co-authors. Here are the top co-authors by number of shared publications:
- Chi-Guhn Lee
University of Toronto
4 shared publications - Kyeonghyeon Park
Department of Industrial and Systems Engineering, Korea Advanced Institute of Science and Technology, Daejeon, Republic of Korea
4 shared publications - Taesik Lee
Department of Industrial and Systems Engineering, Korea Advanced Institute of Science and Technology, Daejeon, Republic of Korea
3 shared publications - David Molina Concha
Engineering Science Faculty of Applied Science and Engineering Complex System Design Laboratory University of Toronto M5S3G8 Toronto ON Canada
2 shared publications - David Molina Concha
Department of Mechanical and Industrial Engineering, University of Toronto, Toronto, Canada
2 shared publications - Taesik Lee
Engineering Science Faculty of Applied Science and Engineering Complex System Design Laboratory University of Toronto M5S3G8 Toronto ON Canada
2 shared publications
Latest Publications
- 29 October 2026
- Published by Elsevier in Simulation Modelling Practice and Theory
- Vol. 154
Automated disaster response scenario generation and simulation for evaluating emergency medical services system
- 29 October 2026
- Published by Elsevier in Simulation Modelling Practice and Theory
- Vol. 154
Automated disaster response scenario generation and simulation for evaluating emergency medical services system
- Open Access
Analyzing Strategic Parental Leave Decisions Using Two-Player Multi-Agent Reinforcement Learning
Despite the well-documented benefits of paid parental leave, many employees hesitate to take it. This study employs a two-player stochastic game (SG) model to analyze how various factors affect parental leave decisions. The proposed SG model incorporates (1) an employee’s perceived utility from taking leave, (2) the effect of colleague’s parental leave, (3) career penalties after taking leave, and (4) a paid parental policy. To accurately obtain equilibrium strategies, we extend Nash-Q learning by incorporating backward iteration and optimistic initialization. These two methods exploit the structural properties of the model to accelerate convergence and improve solution quality. Numerical experiments reveal that a stronger willingness to take parental leave and lower career penalties increase parental leave uptake. Furthermore, the competitive career penalty, which captures interpersonal factors, is particularly influential when a colleague is less likely to take parental leave. Our results suggest that reducing career penalties can substantially increase leave uptake in typical parameter ranges, highlighting the importance of workplace policies that mitigate career penalties associated with parental leave.
Open AccessAnalyzing Strategic Parental Leave Decisions Using Two-Player Multi-Agent Reinforcement Learning
Despite the well-documented benefits of paid parental leave, many employees hesitate to take it. This study employs a two-player stochastic game (SG) model to analyze how various factors affect parental leave decisions. The proposed SG model incorporates (1) an employee’s perceived utility from taking leave, (2) the effect of colleague’s parental leave, (3) career penalties after taking leave, and (4) a paid parental policy. To accurately obtain equilibrium strategies, we extend Nash-Q learning by incorporating backward iteration and optimistic initialization. These two methods exploit the structural properties of the model to accelerate convergence and improve solution quality. Numerical experiments reveal that a stronger willingness to take parental leave and lower career penalties increase parental leave uptake. Furthermore, the competitive career penalty, which captures interpersonal factors, is particularly influential when a colleague is less likely to take parental leave. Our results suggest that reducing career penalties can substantially increase leave uptake in typical parameter ranges, highlighting the importance of workplace policies that mitigate career penalties associated with parental leave.
- Open Access
Hypernetwork-based approach for optimal composition design in partially controlled multi-agent systems
Partially Controlled Multi-Agent Systems (PCMAS) are comprised of controllable agents, managed by a system designer, and uncontrollable agents, operating autonomously. This study addresses an optimal composition design problem in PCMAS, which involves the system designer's problem, determining the optimal number and policies of controllable agents, and the uncontrollable agents' problem, identifying their best-response policies. Solving this bi-level optimization problem is computationally intensive, as it requires repeatedly solving multi-agent reinforcement learning problems under various compositions for both types of agents. To address these challenges, we propose a novel hypernetwork-based framework that jointly optimizes the system's composition and agent policies. Unlike traditional methods that train separate policy networks for each composition, the proposed framework generates policies for both controllable and uncontrollable agents through a unified hypernetwork. This approach enables efficient information sharing across similar configurations, thereby reducing computational overhead. Additional improvements are achieved by incorporating reward parameter optimization and mean action networks. Using real-world New York City taxi data, we demonstrate that our framework outperforms existing methods in approximating equilibrium policies. Our experimental results show significant improvements in key performance metrics, such as order response rate and served demand, highlighting the practical utility of controlling agents and their potential to enhance decision-making in PCMAS.
Open AccessHypernetwork-based approach for optimal composition design in partially controlled multi-agent systems
Partially Controlled Multi-Agent Systems (PCMAS) are comprised of controllable agents, managed by a system designer, and uncontrollable agents, operating autonomously. This study addresses an optimal composition design problem in PCMAS, which involves the system designer's problem, determining the optimal number and policies of controllable agents, and the uncontrollable agents' problem, identifying their best-response policies. Solving this bi-level optimization problem is computationally intensive, as it requires repeatedly solving multi-agent reinforcement learning problems under various compositions for both types of agents. To address these challenges, we propose a novel hypernetwork-based framework that jointly optimizes the system's composition and agent policies. Unlike traditional methods that train separate policy networks for each composition, the proposed framework generates policies for both controllable and uncontrollable agents through a unified hypernetwork. This approach enables efficient information sharing across similar configurations, thereby reducing computational overhead. Additional improvements are achieved by incorporating reward parameter optimization and mean action networks. Using real-world New York City taxi data, we demonstrate that our framework outperforms existing methods in approximating equilibrium policies. Our experimental results show significant improvements in key performance metrics, such as order response rate and served demand, highlighting the practical utility of controlling agents and their potential to enhance decision-making in PCMAS.
- Open Access
- 18 November 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 12, 172256-172265
Bayesian Reinforcement Learning for Adaptive Balancing in an Assembly Line With Human-Robot Collaboration
Reinforcement learning (RL) has been frequently used in recent years to develop intelligent robot agents that collaborate with human workers. In human-robot collaboration (HRC), the adaptation ability of automated robots is critical for efficient collaboration. However, previous studies using RL for assembly lines with HRC did not properly consider the possibilities of dynamics changes in the environment when collaborating with other human workers. In this study, we propose a method for adaptive assembly line balancing to efficiently collaborate with diverse workers having different task proficiencies through Bayesian RL. We mathematically formulate a Hidden-parameter Markov Decision Processes (HiP-MDP) model to represent the task proficiency of unknown workers in sequential assembly tasks. The robotic arm learns generalized policy to collaborate with diverse workers through meta-learning, and in a practical implementation, uses the policy adaptively by estimating the task proficiency of workers through Bayesian inference. We demonstrate the superiority of the proposed method in both virtual and real-world environments. Numerical studies show that the proposed method can collaborate well with diverse new workers.
- 18 November 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 12, 172256-172265
Open AccessBayesian Reinforcement Learning for Adaptive Balancing in an Assembly Line With Human-Robot Collaboration
Reinforcement learning (RL) has been frequently used in recent years to develop intelligent robot agents that collaborate with human workers. In human-robot collaboration (HRC), the adaptation ability of automated robots is critical for efficient collaboration. However, previous studies using RL for assembly lines with HRC did not properly consider the possibilities of dynamics changes in the environment when collaborating with other human workers. In this study, we propose a method for adaptive assembly line balancing to efficiently collaborate with diverse workers having different task proficiencies through Bayesian RL. We mathematically formulate a Hidden-parameter Markov Decision Processes (HiP-MDP) model to represent the task proficiency of unknown workers in sequential assembly tasks. The robotic arm learns generalized policy to collaborate with diverse workers through meta-learning, and in a practical implementation, uses the policy adaptively by estimating the task proficiency of workers through Bayesian inference. We demonstrate the superiority of the proposed method in both virtual and real-world environments. Numerical studies show that the proposed method can collaborate well with diverse new workers.
- Open Access
Bayesian Optimization Framework for Efficient Fleet Design in Autonomous Multi-Robot Exploration
This study addresses the challenge of fleet design optimization in the context of heterogeneous multi-robot fleets, aiming to obtain feasible designs that balance performance and costs. In the domain of autonomous multi-robot exploration, reinforcement learning agents play a central role, offering adaptability to complex terrains and facilitating collaboration among robots. However, modifying the fleet composition results in changes in the learned behavior, and training multi-robot systems using multi-agent reinforcement learning is expensive. Therefore, an exhaustive evaluation of each potential fleet design is infeasible. To tackle these hurdles, we introduce Bayesian Optimization for Fleet Design (BOFD), a framework leveraging multi-objective Bayesian Optimization to explore fleets on the Pareto front of performance and cost while accounting for uncertainty in the design space. Moreover, we establish a sub-linear bound for cumulative regret, supporting BOFD's robustness and efficacy. Extensive benchmark experiments in synthetic and simulated environments demonstrate the superiority of our framework over state-of-the-art methods, achieving efficient fleet designs with minimal fleet evaluations.
Open AccessBayesian Optimization Framework for Efficient Fleet Design in Autonomous Multi-Robot Exploration
This study addresses the challenge of fleet design optimization in the context of heterogeneous multi-robot fleets, aiming to obtain feasible designs that balance performance and costs. In the domain of autonomous multi-robot exploration, reinforcement learning agents play a central role, offering adaptability to complex terrains and facilitating collaboration among robots. However, modifying the fleet composition results in changes in the learned behavior, and training multi-robot systems using multi-agent reinforcement learning is expensive. Therefore, an exhaustive evaluation of each potential fleet design is infeasible. To tackle these hurdles, we introduce Bayesian Optimization for Fleet Design (BOFD), a framework leveraging multi-objective Bayesian Optimization to explore fleets on the Pareto front of performance and cost while accounting for uncertainty in the design space. Moreover, we establish a sub-linear bound for cumulative regret, supporting BOFD's robustness and efficacy. Extensive benchmark experiments in synthetic and simulated environments demonstrate the superiority of our framework over state-of-the-art methods, achieving efficient fleet designs with minimal fleet evaluations.
- Open Access
Algorithmic Contract Design with Reinforcement Learning Agents
We introduce a novel problem setting for algorithmic contract design, named the principal-MARL contract design problem. This setting extends traditional contract design to account for dynamic and stochastic environments using Markov Games and Multi-Agent Reinforcement Learning. To tackle this problem, we propose a Multi-Objective Bayesian Optimization (MOBO) framework named Constrained Pareto Maximum Entropy Search (cPMES). Our approach integrates MOBO and MARL to explore the highly constrained contract design space, identifying promising incentive and recruitment decisions. cPMES transforms the principal-MARL contract design problem into an unconstrained multi-objective problem, leveraging the probability of feasibility as part of the objectives and ensuring promising designs predicted on the feasibility border are included in the Pareto front. By focusing the entropy prediction on designs within the Pareto set, cPMES mitigates the risk of the search strategy being overwhelmed by entropy from constraints. We demonstrate the effectiveness of cPMES through extensive benchmark studies in synthetic and simulated environments, showing its ability to find feasible contract designs that maximize the principal's objectives. Additionally, we provide theoretical support with a sub-linear regret bound concerning the number of iterations.
Open AccessAlgorithmic Contract Design with Reinforcement Learning Agents
We introduce a novel problem setting for algorithmic contract design, named the principal-MARL contract design problem. This setting extends traditional contract design to account for dynamic and stochastic environments using Markov Games and Multi-Agent Reinforcement Learning. To tackle this problem, we propose a Multi-Objective Bayesian Optimization (MOBO) framework named Constrained Pareto Maximum Entropy Search (cPMES). Our approach integrates MOBO and MARL to explore the highly constrained contract design space, identifying promising incentive and recruitment decisions. cPMES transforms the principal-MARL contract design problem into an unconstrained multi-objective problem, leveraging the probability of feasibility as part of the objectives and ensuring promising designs predicted on the feasibility border are included in the Pareto front. By focusing the entropy prediction on designs within the Pareto set, cPMES mitigates the risk of the search strategy being overwhelmed by entropy from constraints. We demonstrate the effectiveness of cPMES through extensive benchmark studies in synthetic and simulated environments, showing its ability to find feasible contract designs that maximize the principal's objectives. Additionally, we provide theoretical support with a sub-linear regret bound concerning the number of iterations.
- 29 April 2024
- Published by Elsevier in European Journal of Operational Research
- Vol. 317 (3) , 1003-1014
Reward function design method to achieve system-level objectives in ambulance diversion problem
- 29 April 2024
- Published by Elsevier in European Journal of Operational Research
- Vol. 317 (3) , 1003-1014
Reward function design method to achieve system-level objectives in ambulance diversion problem
- 1 February 2022
- Published by Elsevier in Computers & Operations Research
- Vol. 138
Skyport location problem for urban air mobility system
- 1 February 2022
- Published by Elsevier in Computers & Operations Research
- Vol. 138
Skyport location problem for urban air mobility system