Nathaniel Hamilton
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Previous Organizations / Institutes
The organizations / institutes from the scholar's other articles in the past ten years
Research Subjects
Nathaniel Hamilton has a total of 32 co-authors. Here are the top co-authors by number of shared publications:
- Taylor Thomas Johnson
Vanderbilt University
5 shared publications - Benedetto Piccoli
Rutgers University - Camden
3 shared publications - Daniel B. Work
Vanderbilt University, USA
3 shared publications - Jonathan Sprinkle
Vanderbilt University, USA
3 shared publications - Maria Laura Delle Monache
Department of Civil and Environmental Engineering, University of California, Berkeley, CA, USA
3 shared publications - Preston K. Robinette
Vanderbilt University
3 shared publications
Latest Publications
- Open Access
- 9 May 2023
- Published by Association for Computing Machinery (ACM)
- p. 110-119
Self-Preserving Genetic Algorithms for Safe Learning in Discrete Action Spaces
Self-Preserving Genetic Algorithms (SPGA) combine the evolutionary strategy of a genetic algorithm with safety assurance methods commonly implemented in safe reinforcement learning (SRL), a branch of reinforcement learning (RL) that accounts for safety in the exploration and decision-making process of the agent. Safe learning approaches are especially important in safety-critical environments, where failure to account for the safety of the controlled system could result in the loss of millions of dollars in hardware or bodily harm to people working nearby, as is true of many cyber-physical systems. While SRL is a viable approach to safe learning, there are many challenges that must be taken into consideration when training agents, such as sample efficiency, stability, and exploration---an issue that is easily addressed by the evolutionary strategy of a genetic algorithm. By combining GAs with the safety mechanisms used with SRL, SPGA offers a safe learning alternative that is able to explore large areas of the solution space, addressing SRL's challenge of exploration. This work implements SPGA with both action masking and run time assurance safety strategies to evolve safe controllers for three types of discrete action space environments applicable to cyber physical systems (control, routing, and operations) and under various safety conditions. Training and testing evaluation metrics are compared with results from SRL trained controllers to validate results. SPGA and SRL controllers are trained across 5 random seeds and evaluated on 500 episodes to calculate average wall time to train, average expected return, and percentage of safe action evaluation metrics. SPGA achieves comparable reward and safety performance results with significantly improved training efficiency (55x faster on average), demonstrating the effectiveness of this safe learning approach.
- 9 May 2023
- Published by Association for Computing Machinery (ACM)
- p. 110-119
Open AccessSelf-Preserving Genetic Algorithms for Safe Learning in Discrete Action Spaces
Self-Preserving Genetic Algorithms (SPGA) combine the evolutionary strategy of a genetic algorithm with safety assurance methods commonly implemented in safe reinforcement learning (SRL), a branch of reinforcement learning (RL) that accounts for safety in the exploration and decision-making process of the agent. Safe learning approaches are especially important in safety-critical environments, where failure to account for the safety of the controlled system could result in the loss of millions of dollars in hardware or bodily harm to people working nearby, as is true of many cyber-physical systems. While SRL is a viable approach to safe learning, there are many challenges that must be taken into consideration when training agents, such as sample efficiency, stability, and exploration---an issue that is easily addressed by the evolutionary strategy of a genetic algorithm. By combining GAs with the safety mechanisms used with SRL, SPGA offers a safe learning alternative that is able to explore large areas of the solution space, addressing SRL's challenge of exploration. This work implements SPGA with both action masking and run time assurance safety strategies to evolve safe controllers for three types of discrete action space environments applicable to cyber physical systems (control, routing, and operations) and under various safety conditions. Training and testing evaluation metrics are compared with results from SRL trained controllers to validate results. SPGA and SRL controllers are trained across 5 random seeds and evaluated on 500 episodes to calculate average wall time to train, average expected return, and percentage of safe action evaluation metrics. SPGA achieves comparable reward and safety performance results with significantly improved training efficiency (55x faster on average), demonstrating the effectiveness of this safe learning approach.
- Open Access
- 9 May 2023
- Published by Association for Computing Machinery (ACM)
- p. 278-279
DEMO: Self-Preserving Genetic Algorithms vs. Safe Reinforcement Learning in Discrete Action Spaces
Safe learning techniques are learning frameworks that take safety into consideration during the training process. Safe reinforcement learning (SRL) combines reinforcement learning (RL) with safety mechanisms such as action masking and run time assurance to protect an agent during the exploration of its environment. This protection, though, can severely hinder an agent's ability to learn optimal policies as the safety systems exacerbate an already difficult exploration challenge for RL agents. An alternative to RL is an optimization approach known as genetic algorithms (GA), which utilize operators that mimic biological evolution to evolve better policies. By combining safety mechanisms with genetic algorithms, this work demonstrates a novel approach to safe learning called Self-Preserving Genetic Algorithms. To highlight the training benefits of SPGA compared to SRL in discrete action spaces, this demonstration trains and deploys an SPGA agent with action masking (SPGA-AM) and an SRL agent with action masking (SRL-AM) in real-time in the CartPole-v0 environment with a safety boundary condition b = 0.75. After training, each of the learned policies are tested in a CartPole-v0 environment with an extended max timesteps value (T = 200 → T = 1000). After the demo, users will have a better understanding of SPGA and SRL training, as well as the benefits of using SPGA to train in discrete action spaces.
- 9 May 2023
- Published by Association for Computing Machinery (ACM)
- p. 278-279
Open AccessDEMO: Self-Preserving Genetic Algorithms vs. Safe Reinforcement Learning in Discrete Action Spaces
Safe learning techniques are learning frameworks that take safety into consideration during the training process. Safe reinforcement learning (SRL) combines reinforcement learning (RL) with safety mechanisms such as action masking and run time assurance to protect an agent during the exploration of its environment. This protection, though, can severely hinder an agent's ability to learn optimal policies as the safety systems exacerbate an already difficult exploration challenge for RL agents. An alternative to RL is an optimization approach known as genetic algorithms (GA), which utilize operators that mimic biological evolution to evolve better policies. By combining safety mechanisms with genetic algorithms, this work demonstrates a novel approach to safe learning called Self-Preserving Genetic Algorithms. To highlight the training benefits of SPGA compared to SRL in discrete action spaces, this demonstration trains and deploys an SPGA agent with action masking (SPGA-AM) and an SRL agent with action masking (SRL-AM) in real-time in the CartPole-v0 environment with a safety boundary condition b = 0.75. After training, each of the learned policies are tested in a CartPole-v0 environment with an extended max timesteps value (T = 200 → T = 1000). After the demo, users will have a better understanding of SPGA and SRL training, as well as the benefits of using SPGA to train in discrete action spaces.
- 1 October 2022
- Published by Springer Nature
- p. 190-206
Training Agents to Satisfy Timed and Untimed Signal Temporal Logic Specifications with Reinforcement Learning
- 1 October 2022
- Published by Springer Nature
- p. 190-206
Training Agents to Satisfy Timed and Untimed Signal Temporal Logic Specifications with Reinforcement Learning
- 23 April 2020
- Published by SPIE-Intl Soc Optical Eng
- Vol. 11425, 114250J
Sonic to knuckles: Evaluations on transfer reinforcement learning
Reinforcement Learning holds the potential to enable many systems with rapid, intelligent automated decision- making. However, reinforcement learning on embodied systems is a much greater challenge than the simulated environments and tasks which have been solved to date. A learner in an embodied system cannot run millions of trials or easily tolerate fatal trajectories. Therefore, the ability to train agents in simulated environments and effectively transfer their knowledge to real-world environments will be crucial, and likely an integral part of constructing future robotic systems. We perform experiments in an original transfer reinforcement learning task we constructed using the game “Sonic 3 and Knuckles," evaluating two transfer learning techniques from the literature.
- 23 April 2020
- Published by SPIE-Intl Soc Optical Eng
- Vol. 11425, 114250J
Sonic to knuckles: Evaluations on transfer reinforcement learning
Reinforcement Learning holds the potential to enable many systems with rapid, intelligent automated decision- making. However, reinforcement learning on embodied systems is a much greater challenge than the simulated environments and tasks which have been solved to date. A learner in an embodied system cannot run millions of trials or easily tolerate fatal trajectories. Therefore, the ability to train agents in simulated environments and effectively transfer their knowledge to real-world environments will be crucial, and likely an integral part of constructing future robotic systems. We perform experiments in an original transfer reinforcement learning task we constructed using the game “Sonic 3 and Knuckles," evaluating two transfer learning techniques from the literature.
- 13 August 2019
- Published by Springer Nature
- p. 160-177
Reachability Analysis for High-Index Linear Differential Algebraic Equations
- 13 August 2019
- Published by Springer Nature
- p. 160-177
Reachability Analysis for High-Index Linear Differential Algebraic Equations
- 1 February 2019
- Published by Elsevier in Transportation Research Part C: Emerging Technologies
- Vol. 99, 82-109
Tracking vehicle trajectories and fuel rates in phantom traffic jams: Methodology and data
- 1 February 2019
- Published by Elsevier in Transportation Research Part C: Emerging Technologies
- Vol. 99, 82-109
Tracking vehicle trajectories and fuel rates in phantom traffic jams: Methodology and data
- Top Cited Papers
- 1 April 2018
- Published by Elsevier in Transportation Research Part C: Emerging Technologies
- Vol. 89, 205-221
Dissipation of stop-and-go waves via control of autonomous vehicles: Field experiments
- 1 April 2018
- Published by Elsevier in Transportation Research Part C: Emerging Technologies
- Vol. 89, 205-221
Top Cited PapersDissipation of stop-and-go waves via control of autonomous vehicles: Field experiments
- Open Access
- 18 April 2017
- Published by Association for Computing Machinery (ACM)
- p. 7
Controlling for Unsafe Events in Dense Traffic through Autonomous Vehicles
This talk focuses on stop-and-go instabilities in dense traffic flows, and how autonomous vehicles can be applied to control for these instabilities.
- 18 April 2017
- Published by Association for Computing Machinery (ACM)
- p. 7
Open AccessControlling for Unsafe Events in Dense Traffic through Autonomous Vehicles
This talk focuses on stop-and-go instabilities in dense traffic flows, and how autonomous vehicles can be applied to control for these instabilities.