Sangjae Bae
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Previous Organizations / Institutes
The organizations / institutes from the scholar's other articles in the past ten years
Research Subjects
Sangjae Bae has a total of 29 co-authors. Here are the top co-authors by number of shared publications:
- Scott Moura
Department of Civil and Environmental Engineering, University of California, Berkeley, CA, USA
5 shared publications Kikuo Fujimura
3 shared publicationsAlireza Nakhaei
2 shared publications- Chiho Choi
Honda Research Institute USA
2 shared publications David Isele
Honda Research Institute, San Jose, CA, USA
2 shared publications- David Isele
Honda Research Institute, San Jose, CA, USA
2 shared publications
Latest Publications
- 20 December 2023
- Published by SAGE Publications in The International Journal of Robotics Research
- Vol. 43 (9) , 1382-1408
Active uncertainty reduction for safe and efficient interaction planning: A shielding-aware dual control approach
The ability to accurately predict others’ behavior is central to the safety and efficiency of robotic systems in interactive settings, such as human–robot interaction and multi-robot teaming tasks. Unfortunately, robots often lack access to key information on which these predictions may hinge, such as other agents’ goals, attention, and willingness to cooperate. Dual control theory addresses this challenge by treating unknown parameters of a predictive model as stochastic hidden states and inferring their values at runtime using information gathered during system operation. While able to optimally and automatically trade off exploration and exploitation, dual control is computationally intractable for general interactive motion planning, mainly due to the fundamental coupling between the robot’s trajectory plan and its prediction of other agents’ intent. In this paper, we present a novel algorithmic approach to enable active uncertainty reduction for interactive motion planning based on the implicit dual control paradigm. Our approach relies on sampling-based approximation of stochastic dynamic programming, leading to a model predictive control problem that can be readily solved by real-time gradient-based optimization methods. The resulting policy is shown to preserve the dual control effect for a broad class of predictive models with both continuous and categorical uncertainty. To ensure the safe operation of the interacting agents, we use a runtime safety filter (also referred to as a “shielding” scheme), which overrides the robot’s dual control policy with a safety fallback strategy when a safety-critical event is imminent. We then augment the dual control framework with an improved variant of the recently proposed shielding-aware robust planning scheme, which proactively balances the nominal planning performance with the risk of high-cost emergency maneuvers triggered by low-probability agent behaviors. We demonstrate the efficacy of our approach with both simulated driving studies and hardware experiments using 1/10 scale autonomous vehicles.
- 20 December 2023
- Published by SAGE Publications in The International Journal of Robotics Research
- Vol. 43 (9) , 1382-1408
Active uncertainty reduction for safe and efficient interaction planning: A shielding-aware dual control approach
The ability to accurately predict others’ behavior is central to the safety and efficiency of robotic systems in interactive settings, such as human–robot interaction and multi-robot teaming tasks. Unfortunately, robots often lack access to key information on which these predictions may hinge, such as other agents’ goals, attention, and willingness to cooperate. Dual control theory addresses this challenge by treating unknown parameters of a predictive model as stochastic hidden states and inferring their values at runtime using information gathered during system operation. While able to optimally and automatically trade off exploration and exploitation, dual control is computationally intractable for general interactive motion planning, mainly due to the fundamental coupling between the robot’s trajectory plan and its prediction of other agents’ intent. In this paper, we present a novel algorithmic approach to enable active uncertainty reduction for interactive motion planning based on the implicit dual control paradigm. Our approach relies on sampling-based approximation of stochastic dynamic programming, leading to a model predictive control problem that can be readily solved by real-time gradient-based optimization methods. The resulting policy is shown to preserve the dual control effect for a broad class of predictive models with both continuous and categorical uncertainty. To ensure the safe operation of the interacting agents, we use a runtime safety filter (also referred to as a “shielding” scheme), which overrides the robot’s dual control policy with a safety fallback strategy when a safety-critical event is imminent. We then augment the dual control framework with an improved variant of the recently proposed shielding-aware robust planning scheme, which proactively balances the nominal planning performance with the risk of high-cost emergency maneuvers triggered by low-probability agent behaviors. We demonstrate the efficacy of our approach with both simulated driving studies and hardware experiments using 1/10 scale autonomous vehicles.
- Open Access
Active Uncertainty Reduction for Safe and Efficient Interaction Planning: A Shielding-Aware Dual Control Approach
The ability to accurately predict others' behavior is central to the safety and efficiency of interactive robotics. Unfortunately, robots often lack access to key information on which these predictions may hinge, such as other agents' goals, attention, and willingness to cooperate. Dual control theory addresses this challenge by treating unknown parameters of a predictive model as stochastic hidden states and inferring their values at runtime using information gathered during system operation. While able to optimally and automatically trade off exploration and exploitation, dual control is computationally intractable for general interactive motion planning. In this paper, we present a novel algorithmic approach to enable active uncertainty reduction for interactive motion planning based on the implicit dual control paradigm. Our approach relies on sampling-based approximation of stochastic dynamic programming, leading to a model predictive control problem that can be readily solved by real-time gradient-based optimization methods. The resulting policy is shown to preserve the dual control effect for a broad class of predictive models with both continuous and categorical uncertainty. To ensure the safe operation of the interacting agents, we use a runtime safety filter (also referred to as a "shielding" scheme), which overrides the robot's dual control policy with a safety fallback strategy when a safety-critical event is imminent. We then augment the dual control framework with an improved variant of the recently proposed shielding-aware robust planning scheme, which proactively balances the nominal planning performance with the risk of high-cost emergency maneuvers triggered by low-probability agent behaviors. We demonstrate the efficacy of our approach with both simulated driving studies and hardware experiments using 1/10 scale autonomous vehicles.
Open AccessActive Uncertainty Reduction for Safe and Efficient Interaction Planning: A Shielding-Aware Dual Control Approach
The ability to accurately predict others' behavior is central to the safety and efficiency of interactive robotics. Unfortunately, robots often lack access to key information on which these predictions may hinge, such as other agents' goals, attention, and willingness to cooperate. Dual control theory addresses this challenge by treating unknown parameters of a predictive model as stochastic hidden states and inferring their values at runtime using information gathered during system operation. While able to optimally and automatically trade off exploration and exploitation, dual control is computationally intractable for general interactive motion planning. In this paper, we present a novel algorithmic approach to enable active uncertainty reduction for interactive motion planning based on the implicit dual control paradigm. Our approach relies on sampling-based approximation of stochastic dynamic programming, leading to a model predictive control problem that can be readily solved by real-time gradient-based optimization methods. The resulting policy is shown to preserve the dual control effect for a broad class of predictive models with both continuous and categorical uncertainty. To ensure the safe operation of the interacting agents, we use a runtime safety filter (also referred to as a "shielding" scheme), which overrides the robot's dual control policy with a safety fallback strategy when a safety-critical event is imminent. We then augment the dual control framework with an improved variant of the recently proposed shielding-aware robust planning scheme, which proactively balances the nominal planning performance with the risk of high-cost emergency maneuvers triggered by low-probability agent behaviors. We demonstrate the efficacy of our approach with both simulated driving studies and hardware experiments using 1/10 scale autonomous vehicles.
- Open Access
Driving in Dense Traffic with Model-Free Reinforcement Learning
Traditional planning and control methods could fail to find a feasible trajectory for an autonomous vehicle to execute amongst dense traffic on roads. This is because the obstacle-free volume in spacetime is very small in these scenarios for the vehicle to drive through. However, that does not mean the task is infeasible since human drivers are known to be able to drive amongst dense traffic by leveraging the cooperativeness of other drivers to open a gap. The traditional methods fail to take into account the fact that the actions taken by an agent affect the behaviour of other vehicles on the road. In this work, we rely on the ability of deep reinforcement learning to implicitly model such interactions and learn a continuous control policy over the action space of an autonomous vehicle. The application we consider requires our agent to negotiate and open a gap in the road in order to successfully merge or change lanes. Our policy learns to repeatedly probe into the target road lane while trying to find a safe spot to move in to. We compare against two model-predictive control-based algorithms and show that our policy outperforms them in simulation.
Open AccessDriving in Dense Traffic with Model-Free Reinforcement Learning
Traditional planning and control methods could fail to find a feasible trajectory for an autonomous vehicle to execute amongst dense traffic on roads. This is because the obstacle-free volume in spacetime is very small in these scenarios for the vehicle to drive through. However, that does not mean the task is infeasible since human drivers are known to be able to drive amongst dense traffic by leveraging the cooperativeness of other drivers to open a gap. The traditional methods fail to take into account the fact that the actions taken by an agent affect the behaviour of other vehicles on the road. In this work, we rely on the ability of deep reinforcement learning to implicitly model such interactions and learn a continuous control policy over the action space of an autonomous vehicle. The application we consider requires our agent to negotiate and open a gap in the road in order to successfully merge or change lanes. Our policy learns to repeatedly probe into the target road lane while trying to find a safe spot to move in to. We compare against two model-predictive control-based algorithms and show that our policy outperforms them in simulation.
- 3 August 2022
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Control Systems Technology
- Vol. 31 (2) , 646-659
Lane-Change in Dense Traffic With Model Predictive Control and Neural Networks
This article presents an online smooth-path lane-change control framework. We focus on dense traffic where intervehicle space gaps are narrow, and cooperation with surrounding drivers is essential to achieve the lane-change maneuver. We propose a two-stage control framework that harmonizes model predictive control (MPC) with generative adversarial networks (GANs) by utilizing driving intentions to generate smooth lane-change maneuvers. To improve performance in practice, the system is augmented with an adaptive safety boundary and a Kalman filter to mitigate sensor noise. Simulation studies are investigated at different levels of traffic density and cooperativeness of other drivers. The simulation results support the effectiveness, driving comfort, and safety of the proposed method.
- 3 August 2022
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Control Systems Technology
- Vol. 31 (2) , 646-659
Lane-Change in Dense Traffic With Model Predictive Control and Neural Networks
This article presents an online smooth-path lane-change control framework. We focus on dense traffic where intervehicle space gaps are narrow, and cooperation with surrounding drivers is essential to achieve the lane-change maneuver. We propose a two-stage control framework that harmonizes model predictive control (MPC) with generative adversarial networks (GANs) by utilizing driving intentions to generate smooth lane-change maneuvers. To improve performance in practice, the system is augmented with an adaptive safety boundary and a Kalman filter to mitigate sensor noise. Simulation studies are investigated at different levels of traffic density and cooperativeness of other drivers. The simulation results support the effectiveness, driving comfort, and safety of the proposed method.
- 27 January 2022
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Robotics and Automation Letters
- Vol. 7 (2) , 3194-3201
Spatiotemporal Costmap Inference for MPC Via Deep Inverse Reinforcement Learning
It can be difficult to autonomously produce driver behavior so that it appears natural to other traffic participants. Through Inverse Reinforcement Learning (IRL), we can automate this process by learning the underlying reward function from human demonstrations. We propose a new IRL algorithm that learns a goal-conditioned spatio-temporal reward function. The resulting costmap is used by Model Predictive Controllers (MPCs) to perform a task without any hand-designing or hand-tuning of the cost function. We evaluate our proposed Goal-conditioned SpatioTemporal Zeroing Maximum Entropy Deep IRL (GSTZ)-MEDIRL framework together with MPC in the CARLA simulator for autonomous driving, lane keeping, and lane changing tasks in a challenging dense traffic highway scenario. Our proposed methods show higher success rates compared to other baseline methods including behavior cloning, state-of-the-art RL policies, and MPC with a learning-based behavior prediction model.
- 27 January 2022
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Robotics and Automation Letters
- Vol. 7 (2) , 3194-3201
Spatiotemporal Costmap Inference for MPC Via Deep Inverse Reinforcement Learning
It can be difficult to autonomously produce driver behavior so that it appears natural to other traffic participants. Through Inverse Reinforcement Learning (IRL), we can automate this process by learning the underlying reward function from human demonstrations. We propose a new IRL algorithm that learns a goal-conditioned spatio-temporal reward function. The resulting costmap is used by Model Predictive Controllers (MPCs) to perform a task without any hand-designing or hand-tuning of the cost function. We evaluate our proposed Goal-conditioned SpatioTemporal Zeroing Maximum Entropy Deep IRL (GSTZ)-MEDIRL framework together with MPC in the CARLA simulator for autonomous driving, lane keeping, and lane changing tasks in a challenging dense traffic highway scenario. Our proposed methods show higher success rates compared to other baseline methods including behavior cloning, state-of-the-art RL policies, and MPC with a learning-based behavior prediction model.
- 1 January 2022
- Published by ASME International in Journal of Dynamic Systems, Measurement, and Control
- Vol. 144 (1)
Ecological Adaptive Cruise Control of Plug-In Hybrid Electric Vehicle With Connected Infrastructure and On-Road Experiments
This paper examines both mathematical formulation and practical implementation of an ecological adaptive cruise controller (ECO-ACC) with connected infrastructure. Human errors are typical sources of accidents in urban driving, which can be remedied by rigorous control theories. Designing an ECO-ACC is, therefore, a classical research problem to improve safety and energy efficiency. We add two main contributions to the literature. First, we propose a mathematical framework of an online ECO-ACC for plug-in hybrid electric vehicle (PHEV). Second, we demonstrate ECO-ACC in a real world, which includes other human drivers and uncertain traffic signals on a 2.6 (km) length of the corridor with eight signalized intersections in Southern California. The demonstration results show, on average, 30.98% of energy efficiency improvement and 8.51% additional travel time.
- 1 January 2022
- Published by ASME International in Journal of Dynamic Systems, Measurement, and Control
- Vol. 144 (1)
Ecological Adaptive Cruise Control of Plug-In Hybrid Electric Vehicle With Connected Infrastructure and On-Road Experiments
This paper examines both mathematical formulation and practical implementation of an ecological adaptive cruise controller (ECO-ACC) with connected infrastructure. Human errors are typical sources of accidents in urban driving, which can be remedied by rigorous control theories. Designing an ECO-ACC is, therefore, a classical research problem to improve safety and energy efficiency. We add two main contributions to the literature. First, we propose a mathematical framework of an online ECO-ACC for plug-in hybrid electric vehicle (PHEV). Second, we demonstrate ECO-ACC in a real world, which includes other human drivers and uncertain traffic signals on a 2.6 (km) length of the corridor with eight signalized intersections in Southern California. The demonstration results show, on average, 30.98% of energy efficiency improvement and 8.51% additional travel time.
- 1 May 2021
- Published by Elsevier in Applied Energy
- Vol. 290
Pareto optimality in cost and service quality for an Electric Vehicle charging facility
- 1 May 2021
- Published by Elsevier in Applied Energy
- Vol. 290
Pareto optimality in cost and service quality for an Electric Vehicle charging facility
- 18 March 2021
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Smart Grid
- Vol. 12 (4) , 3353-3363
Inducing Human Behavior to Maximize Operation Performance at PEV Charging Station
Plug-in electric vehicle (PEV) charging station service capability is physically limited by the charger availability and local transformer capacity. However, the station operation performance has become an increasingly important factor for enhancing charging service accessibility. In this work, we propose an innovative station-level optimization framework to operate charging station with optimal pricing policy and charge scheduling. This model incorporates human behaviors to explicitly and effectively capture drivers' charging decision process. We propose a menu of price-differentiated charging services, which differ in both the per-unit prices and the energy delivery schedule. Involving human in the loop, the operation model also exploits the capability to alleviate the overstay issue that occurs when a PEV's charging session has completed. We then propose a muli-block convex transformation methodology to reformulate the resulted non-convex problem via Fenchel-Young Inequality; the Block Coordinate Descent algorithm is applied to solve the overall problem and the efficiency enables real-time implementation. As a result, our simulation demonstrates that the proposed control polices can realize benefits in three aspects: (i) net profits gains, (ii) overstay reduction, and (iii) quality-of-service increase.
- 18 March 2021
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Smart Grid
- Vol. 12 (4) , 3353-3363
Inducing Human Behavior to Maximize Operation Performance at PEV Charging Station
Plug-in electric vehicle (PEV) charging station service capability is physically limited by the charger availability and local transformer capacity. However, the station operation performance has become an increasingly important factor for enhancing charging service accessibility. In this work, we propose an innovative station-level optimization framework to operate charging station with optimal pricing policy and charge scheduling. This model incorporates human behaviors to explicitly and effectively capture drivers' charging decision process. We propose a menu of price-differentiated charging services, which differ in both the per-unit prices and the energy delivery schedule. Involving human in the loop, the operation model also exploits the capability to alleviate the overstay issue that occurs when a PEV's charging session has completed. We then propose a muli-block convex transformation methodology to reformulate the resulted non-convex problem via Fenchel-Young Inequality; the Block Coordinate Descent algorithm is applied to solve the overall problem and the efficiency enables real-time implementation. As a result, our simulation demonstrates that the proposed control polices can realize benefits in three aspects: (i) net profits gains, (ii) overstay reduction, and (iii) quality-of-service increase.
- Open Access
Cooperation-Aware Lane Change Maneuver in Dense Traffic based on Model Predictive Control with Recurrent Neural Network
This paper presents a real-time lane change control framework of autonomous driving in dense traffic, which exploits cooperative behaviors of other drivers. This paper focuses on heavy traffic where vehicles cannot change lanes without cooperating with other drivers. In this case, classical robust controls may not apply since there is no safe area to merge to without interacting with the other drivers. That said, modeling complex and interactive human behaviors is highly non-trivial from the perspective of control engineers. We propose a mathematical control framework based on Model Predictive Control (MPC) encompassing a state-of-the-art Recurrent Neural network (RNN) architecture. In particular, RNN predicts interactive motions of other drivers in response to potential actions of the autonomous vehicle, which are then systematically evaluated in safety constraints. We also propose a real-time heuristic algorithm to find locally optimal control inputs. Finally, quantitative and qualitative analysis on simulation studies are presented to illustrate the benefits of the proposed framework.
Open AccessCooperation-Aware Lane Change Maneuver in Dense Traffic based on Model Predictive Control with Recurrent Neural Network
This paper presents a real-time lane change control framework of autonomous driving in dense traffic, which exploits cooperative behaviors of other drivers. This paper focuses on heavy traffic where vehicles cannot change lanes without cooperating with other drivers. In this case, classical robust controls may not apply since there is no safe area to merge to without interacting with the other drivers. That said, modeling complex and interactive human behaviors is highly non-trivial from the perspective of control engineers. We propose a mathematical control framework based on Model Predictive Control (MPC) encompassing a state-of-the-art Recurrent Neural network (RNN) architecture. In particular, RNN predicts interactive motions of other drivers in response to potential actions of the autonomous vehicle, which are then systematically evaluated in safety constraints. We also propose a real-time heuristic algorithm to find locally optimal control inputs. Finally, quantitative and qualitative analysis on simulation studies are presented to illustrate the benefits of the proposed framework.
- Open Access
Real-time Ecological Velocity Planning for Plug-in Hybrid Vehicles with Partial Communication to Traffic Lights
This paper presents the design of an ecological adaptive cruise controller (ECO-ACC) for a plug-in hybrid vehicle (PHEV) which exploits automated driving and connectivity. Most existing papers for ECO-ACC focus on a short-sighted control scheme. A two-level control framework for long-sighted ECO-ACC was only recently introduced. However, that work is based on a deterministic traffic signal phase and timing (SPaT) over the entire route. In practice, connectivity with traffic lights may be limited by communication range, e.g. just one upcoming traffic light. We propose a two-level receding-horizon control framework for long-sighted ECO-ACC that exploits deterministic SPaT for the upcoming traffic light, and utilizes historical SPaT for other traffic lights within a receding control horizon. We also incorporate a powertrain control mechanism to enhance PHEV energy prediction accuracy. Hardware-in-the-loop simulation results validate the energy savings of the receding-horizon control framework in various traffic scenarios.
Open AccessReal-time Ecological Velocity Planning for Plug-in Hybrid Vehicles with Partial Communication to Traffic Lights
This paper presents the design of an ecological adaptive cruise controller (ECO-ACC) for a plug-in hybrid vehicle (PHEV) which exploits automated driving and connectivity. Most existing papers for ECO-ACC focus on a short-sighted control scheme. A two-level control framework for long-sighted ECO-ACC was only recently introduced. However, that work is based on a deterministic traffic signal phase and timing (SPaT) over the entire route. In practice, connectivity with traffic lights may be limited by communication range, e.g. just one upcoming traffic light. We propose a two-level receding-horizon control framework for long-sighted ECO-ACC that exploits deterministic SPaT for the upcoming traffic light, and utilizes historical SPaT for other traffic lights within a receding control horizon. We also incorporate a powertrain control mechanism to enhance PHEV energy prediction accuracy. Hardware-in-the-loop simulation results validate the energy savings of the receding-horizon control framework in various traffic scenarios.