PLPengyuan Lu

Pengyuan Lu

*This is an algorithmically generated author record based on publicly available information.

Current Organizations / Institutes

The organizations/institutes from the scholar's latest article


Pengyuan Lu has a total of 10 co-authors. Here are the top co-authors by number of shared publications:


Latest Publications

  • Open Access

    Deadline-Safe Control Synthesis for Cyber-Physical Systems with Reinforcement Learning

    by Mengyu Liu, Pengyuan Lu, Shixiong Jiang, Xin Chen, Oleg Sokolsky, Insup Lee, Fanxin Kong

    Cyber-physical systems (CPS) play a pivotal role in industrial automation, transportation, and critical infrastructure, where meeting stringent timing constraints is essential to ensure operational safety and efficiency. While reinforcement learning (RL) has shown promise in synthesizing controllers for time-critical applications, existing approaches often prioritize speed (as soon as possible, or ASAP) without explicitly addressing deadline compliance. This misalignment can lead to unsafe or suboptimal behaviors, which are unacceptable in industrial contexts requiring both safety and reliability under hard deadlines. For example, with inappropriate rewards, a control policy for a industrial robot can be encouraged to be less safe but fast instead of being steady and meeting the deadlines. To address this challenge, we investigate the relationship between ASAP behavior and deadline-safe behavior, introducing a novel Markov decision process formulation (R-MDP) that includes time-awareness while preserving the Markov property. We propose a reward design method that systematically encourages deadline compliance and guarantees safety in reach-avoid tasks. Our approach is validated on multiple benchmarks, including linear and nonlinear systems representative of industrial applications, such as DC motor control and real-time attitude control. Experimental results demonstrate the efficacy of our method in achieving deadline-safe control while maintaining system safety, offering a reliable solution for industrial CPS where failure to meet deadlines can have severe consequences.

  • Open Access

    Repairing neural network-based control policies with safety preservation

    by Pengyuan Lu, Matthew Cleaveland, Oleg Sokolsky, Insup Lee, Ivan Ruchkin

    Neural network (NN)-based control policies have proven their advantages in cyber-physical systems (CPS). When an NN-based policy fails to fulfill a formal specification, engineers leverage NN repair algorithms to fix its behaviors. However, such repair techniques risk breaking the existing correct behaviors, losing not only correctness but also verifiability of initial state subsets. That is, the repair may introduce new risks, previously unaccounted for. In response, we formalize the problem of Repair with Preservation (RwP) and develop Incremental Simulated Annealing Repair (ISAR). ISAR is an NN repair algorithm that aims to preserve correctness and verifiability—while repairing as many failures as possible. Our algorithm leverages simulated annealing on a barriered energy function to safeguard the already-correct initial states while repairing as many additional ones as possible. Moreover, formal verification is utilized to guarantee the repair results. ISAR is compared to a reviewed set of state-of-the-art algorithms, including (1) reinforcement learning-based techniques (STLGym and F-MDP), (2) supervised learning-based techniques (MIQP and minimally deviating repair) and (3) online shielding techniques (tube MPC shielding). Upon evaluation on two standard benchmarks, OpenAI Gym mountain car and an unmanned underwater vehicle, ISAR not only preserves correct behaviors from previously verified initial state regions, but also repairs 81.4% and 23.5% of broken state spaces in the two benchmarks. Moreover, the signal temporal logic (STL) robustness of the ISAR-repaired policies is higher than the baselines.

  • Open Access

    Recovery from Adversarial Attacks in Cyber-physical Systems: Shallow, Deep, and Exploratory Works

    by Pengyuan Lu, Lin Zhang, Mengyu Liu, Kaustubh Sridhar, Oleg Sokolsky, Fanxin Kong, Insup Lee

    Cyber-physical systems (CPS) have experienced rapid growth in recent decades. However, like any other computer-based systems, malicious attacks evolve mutually, driving CPS to undesirable physical states, and potentially causing catastrophes. Although the current state-of-the-art is well aware of this issue, the majority of researchers have not focused on CPS recovery, the procedure we defined as restoring a CPS’s physical state back to a target condition under adversarial attacks. To call for attention on CPS recovery and identify existing efforts, we have surveyed a total of 30 relevant papers. We identify a major partition of the proposed recovery strategies: shallow recovery vs. deep recovery, where the former does not use a dedicated recovery controller while the latter does. Additionally, we surveyed exploratory research on topics that facilitate recovery. From these publications, we discuss the current state-of-the-art of CPS recovery, with respect to applications, attack type, attack surfaces, and system dynamics. Then, we identify untouched sub-domains in this field and suggest possible future directions for researchers.