Jiyoung Chang
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Research Subjects
Jiyoung Chang has a total of 4 co-authors. Here are the top co-authors by number of shared publications:
- Gihwon Kwon
Department of Computer Science, Kyonggi University, Yeongtong-gu, Suwon-si, Gyeonggi-do, Republic of Korea
4 shared publications - Ryeonggu Kwon
Department of Computer Science, Kyonggi University, Yeongtong-gu, Suwon-si, Gyeonggi-do, Republic of Korea
3 shared publications - Sohee Park
Department of SW Safety and Cyber Security, Kyonggi University, Suwon-si 16227, Gyeonggi-do, Republic of Korea
1 shared publication - Suhee Jo
Department of SW Safety and Security, Kyonggi University, Suwon-si 154-42, Gyeonggi-do, Republic of Korea
1 shared publication
Latest Publications
- Open Access
- 29 March 2024
- Published by MDPI AG in Applied Sciences
- Vol. 14 (7) , 2916
STPA-RL: Integrating Reinforcement Learning into STPA for Loss Scenario Exploration
Experience-based methods like reinforcement learning (RL) are often deemed less suitable for the safety field due to concerns about potential safety issues. To bridge this gap, we introduce STPA-RL, a methodology that integrates RL with System-Theoretic Process Analysis (STPA). STPA is a safety analysis technique that identifies causative factors leading to unsafe control actions and system hazards through loss scenarios. In the context of STPA-RL, we formalize the Markov Decision Process based on STPA analysis results to incorporate control algorithms into the system environment. The agent learns safe actions through reward-based learning, tracking potential hazard paths to validate system safety. Specifically, by analyzing various loss scenarios related to the Platform Screen Door, we assess the applicability of the proposed approach by evaluating hazard trajectory graphs and hazard frequencies in the system. This paper streamlines the RL process for loss scenario identification through STPA, contributing to self-guided loss scenarios and diverse system modeling. Additionally, it offers effective simulations for proactive development to enhance system safety and provide practical assistance in the safety field.
- 29 March 2024
- Published by MDPI AG in Applied Sciences
- Vol. 14 (7) , 2916
Open AccessSTPA-RL: Integrating Reinforcement Learning into STPA for Loss Scenario Exploration
Experience-based methods like reinforcement learning (RL) are often deemed less suitable for the safety field due to concerns about potential safety issues. To bridge this gap, we introduce STPA-RL, a methodology that integrates RL with System-Theoretic Process Analysis (STPA). STPA is a safety analysis technique that identifies causative factors leading to unsafe control actions and system hazards through loss scenarios. In the context of STPA-RL, we formalize the Markov Decision Process based on STPA analysis results to incorporate control algorithms into the system environment. The agent learns safe actions through reward-based learning, tracking potential hazard paths to validate system safety. Specifically, by analyzing various loss scenarios related to the Platform Screen Door, we assess the applicability of the proposed approach by evaluating hazard trajectory graphs and hazard frequencies in the system. This paper streamlines the RL process for loss scenario identification through STPA, contributing to self-guided loss scenarios and diverse system modeling. Additionally, it offers effective simulations for proactive development to enhance system safety and provide practical assistance in the safety field.
- Open Access
- 25 January 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 12, 18957-18971
Applying Quantitative Model Checking to Analyze Safety in Reinforcement Learning
Reinforcement learning (RL) is rapidly used in safety-centric applications. However, many studies focus on generating optimal policy that achieves maximum rewards. While maximum rewards are beneficial, safety constraints and non-functional requirements must also be considered in safety-centric applications to avoid dangerous situations. For example, in the case of food delivery robots in restaurants, RL should be used not only to find optimal policy that response to all customer requests through maximum rewards but also to consider safety constraints such as collision avoidance and non-functional requirements such as battery saving. In this paper, we investigated the fulfillment of safety constraints and non-functional requirements of learning models generated through RL with quantitative model checking. We experimented with various time steps and learning rates required for RL, targeting restaurant delivery robots. The functional requirement of these robots is to process all customer order requests, and the non-functional requirements are the number of steps and battery consumption to complete the task. Safety constraints include the amount of collision and the probability of collision. Through these experiments, we made three important findings. First, learning models that obtain maximum rewards may have a low degree of achievement of non-functional requirements and safety constraints. Second, as safety constraints are met, the degree of achievement of non-functional requirements may be low. Third, even if the maximum reward is not obtained, sacrificing non-functional requirements can maximize the achievement of safety constraints. These results show that learning models generated through RL can trade off rewards to achieve safety constraints. In conclusion, our work can contribute to selecting suitable hyperparameters and optimal learning models during RL.
- 25 January 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 12, 18957-18971
Open AccessApplying Quantitative Model Checking to Analyze Safety in Reinforcement Learning
Reinforcement learning (RL) is rapidly used in safety-centric applications. However, many studies focus on generating optimal policy that achieves maximum rewards. While maximum rewards are beneficial, safety constraints and non-functional requirements must also be considered in safety-centric applications to avoid dangerous situations. For example, in the case of food delivery robots in restaurants, RL should be used not only to find optimal policy that response to all customer requests through maximum rewards but also to consider safety constraints such as collision avoidance and non-functional requirements such as battery saving. In this paper, we investigated the fulfillment of safety constraints and non-functional requirements of learning models generated through RL with quantitative model checking. We experimented with various time steps and learning rates required for RL, targeting restaurant delivery robots. The functional requirement of these robots is to process all customer order requests, and the non-functional requirements are the number of steps and battery consumption to complete the task. Safety constraints include the amount of collision and the probability of collision. Through these experiments, we made three important findings. First, learning models that obtain maximum rewards may have a low degree of achievement of non-functional requirements and safety constraints. Second, as safety constraints are met, the degree of achievement of non-functional requirements may be low. Third, even if the maximum reward is not obtained, sacrificing non-functional requirements can maximize the achievement of safety constraints. These results show that learning models generated through RL can trade off rewards to achieve safety constraints. In conclusion, our work can contribute to selecting suitable hyperparameters and optimal learning models during RL.
- 31 July 2023
- Published by Korean Institute of Information Technology in The Journal of Korean Institute of Information Technology
- Vol. 21 (7) , 39-48
STPA-RL: Analyzing Loss Scenarios in STPA with Reinforcement Learning
시스템의 위험 분석 기법인 STPA(System Theoretic Process Analysis)에서는 안전하지 않은 제어 행동(Unsafe Control Action)으로 야기된 위험의 원인과 결과 관계를 설명하는 손실 시나리오 식별이 필수적이다. 지금까지는 전문가의 수작업, 주관적, 비체계적인 방법으로 손실 시나리오를 식별하였다. 이에 본 논문에서는 강화학습(RL, Reinforcement Learning)을 결합하여, 손실 시나리오를 자동으로 도출하는 STPA-RL을 제안한다. 이를 위하여 STPA 분석 결과를 바탕으로 안전한 제어 행동을 강화 학습하는 환경을 모델링하고, 이어서 위험상태에 도달하는 시스템 상태 전이 과정을 탐색한다. 산업 공정 시스템을 사례연구로 실험한 결과, 약 400개의 손실 시나리오를 생성할 수 있었다. 그 결과, 높은 빈도를 갖는 위험을 파악할 수 있었을 뿐만 아니라 위험에 이르는 상태 변화 과정을 시각화하여 손실 시나리오 이해를 높일 수 있었다.
- 31 July 2023
- Published by Korean Institute of Information Technology in The Journal of Korean Institute of Information Technology
- Vol. 21 (7) , 39-48
STPA-RL: Analyzing Loss Scenarios in STPA with Reinforcement Learning
시스템의 위험 분석 기법인 STPA(System Theoretic Process Analysis)에서는 안전하지 않은 제어 행동(Unsafe Control Action)으로 야기된 위험의 원인과 결과 관계를 설명하는 손실 시나리오 식별이 필수적이다. 지금까지는 전문가의 수작업, 주관적, 비체계적인 방법으로 손실 시나리오를 식별하였다. 이에 본 논문에서는 강화학습(RL, Reinforcement Learning)을 결합하여, 손실 시나리오를 자동으로 도출하는 STPA-RL을 제안한다. 이를 위하여 STPA 분석 결과를 바탕으로 안전한 제어 행동을 강화 학습하는 환경을 모델링하고, 이어서 위험상태에 도달하는 시스템 상태 전이 과정을 탐색한다. 산업 공정 시스템을 사례연구로 실험한 결과, 약 400개의 손실 시나리오를 생성할 수 있었다. 그 결과, 높은 빈도를 갖는 위험을 파악할 수 있었을 뿐만 아니라 위험에 이르는 상태 변화 과정을 시각화하여 손실 시나리오 이해를 높일 수 있었다.
- 30 November 2022
- Published by Korean Institute of Information Technology in The Journal of Korean Institute of Information Technology
- Vol. 20 (11) , 65-72
Analysis of Hyper-parameters in Solving Sokoban using Q-learning
본 논문에서는 Q-학습을 이용하여 소코반을 풀이한다. Q-학습은 환경의 현재 상태에 따라 무작위의 행동을 선택하여 탐험을 진행하거나 학습의 활용을 통하여 최대의 보상을 얻을 수 있는 행동을 선택한다. 최적의 Q-테이블을 만들기 위한 과정에서 학습에 영향을 미치는 학습율, 할인율 그리고 감소율 3가지의 하이퍼-파라미터 값을 모든 조합으로 변경하며 학습 성공률을 비교 분석하였다. 이를 이용해 하이퍼-파라미터 값을 각각 학습율은 0.3, 할인율은 0.9 그리고 감소율은 0.9로 고정하고 하나의 값만을 0에서부터 1까지 0.01씩 변경하며 소코반 풀이에 성공과 실패를 결정하는 Q-테이블에 대한 평가를 진행하였다. 그 결과, 소코반 학습과 풀이 평가에 가장 좋은 학습 결과를 제공할 수 있는 하이퍼-파라미터 값을 도출하였다. 감소율의 값이 0.2 부근 일 때 학습성공률이 가장 컸으며, 감소율을 0.96으로 지정했을 때 최적의 풀이에 가까운 학습에 성공하였다.
- 30 November 2022
- Published by Korean Institute of Information Technology in The Journal of Korean Institute of Information Technology
- Vol. 20 (11) , 65-72
Analysis of Hyper-parameters in Solving Sokoban using Q-learning
본 논문에서는 Q-학습을 이용하여 소코반을 풀이한다. Q-학습은 환경의 현재 상태에 따라 무작위의 행동을 선택하여 탐험을 진행하거나 학습의 활용을 통하여 최대의 보상을 얻을 수 있는 행동을 선택한다. 최적의 Q-테이블을 만들기 위한 과정에서 학습에 영향을 미치는 학습율, 할인율 그리고 감소율 3가지의 하이퍼-파라미터 값을 모든 조합으로 변경하며 학습 성공률을 비교 분석하였다. 이를 이용해 하이퍼-파라미터 값을 각각 학습율은 0.3, 할인율은 0.9 그리고 감소율은 0.9로 고정하고 하나의 값만을 0에서부터 1까지 0.01씩 변경하며 소코반 풀이에 성공과 실패를 결정하는 Q-테이블에 대한 평가를 진행하였다. 그 결과, 소코반 학습과 풀이 평가에 가장 좋은 학습 결과를 제공할 수 있는 하이퍼-파라미터 값을 도출하였다. 감소율의 값이 0.2 부근 일 때 학습성공률이 가장 컸으며, 감소율을 0.96으로 지정했을 때 최적의 풀이에 가까운 학습에 성공하였다.