Rongjie Yan

Background: Object detection modules are essential functionalities for any autonomous vehicle. However, the performance of such modules implemented using deep neural networks can be unreliable in many cases, which raises the necessity to filter the abnormal outputs for safety considerations. Aims: This article aims to develop a logical framework for filtering potentially erroneous object detection results. Materials & Methods: Concretely, we consider two types of abstraction, namely spatial abstraction and temporal abstraction, based on the data labels from the training dataset of object detectors, and temporal consistency between a sequence of images. Operated on the training dataset, the construction of spatial abstraction iterates each input, aggregates region‐wise information over its associated labels, and stores the object abstraction. The abstraction is adopted to filter static and spatial abnormalities. The temporal abstraction builds an abstract transformer for a relaxed tracking algorithm. Elements being associated together by the abstract transformer can be checked against consistency over their original values. The abstraction helps to monitor temporal abnormality in consecutive frames. We have implemented the overall framework and validated it using publicly available datasets and open‐source object detectors. Results: The implemented framework successfully identified most of static/spatial and temporal abnormalities in object detection outputs. Validation on public datasets confirmed the effectiveness of the abstraction‐based approach in filtering unreliable detections. Discussion: The results demonstrate that logical abstractions derived from training data labels and temporal sequences provide a viable mechanism for monitoring the reliability of deep neural network‐based object detectors. Conclusion: Abstraction‐based monitoring presents a robust and logical framework for enhancing the reliability of object detectors by filtering abnormal detection results.

Autonomous surgery involves surgical tasks performed by a robot with minimal or no human involvement. Thanks to its precise automation, surgical robotics offers significant benefits in enhancing the consistency, safety, and quality of procedures, driving its growing popularity. However, ensuring the safety of autonomous surgical robotic systems remains a significant challenge. To address this, we propose a simulation-based validation method to detect potential safety issues in the software of surgical robotic systems, complemented by a digital twin to estimate the gap between simulation and reality. The validation framework consists of a test case generator and a monitor for validating properties and evaluating the performance of the robotic system during test execution. Using a robotic arm for needle insertion as a case study, we present a systematic test case generation method that ensures effective coverage measurement for a three-dimensional, irregular model. Since no simulation can perfectly replicate reality due to differences in sensing and actuation, the digital twin bridges the gap between simulation and the physical robotic arm. This integration enables us to assess the discrepancy between virtual simulations and real-world operations by verifying whether the data from the simulation accurately predicts real-world outcomes. Through extensive experimentation, we identified several flaws in the robotic software. Co-simulation within the digital twin framework has highlighted these discrepancies that should be considered.

Deep neural networks (DNNs) are instrumental in realizing complex perception systems. As many of these applications are safety-critical by design, engineering rigor is required to ensure that the functional insufficiency of the DNN-based perception is not the source of harm. In addition to conventional static verification and testing techniques employed during the design phase, there is a need for runtime verification techniques that can detect critical events, diagnose issues, and even enforce requirements. This tutorial aims to provide readers with a glimpse of techniques proposed in the literature. We start with classical methods proposed in the machine learning community, then highlight a few techniques proposed by the formal methods community. While we surely can observe similarities in the design of monitors, how the decision boundaries are created vary between the two communities. We conclude by highlighting the need to rigorously design monitors, where data availability outside the operational domain plays an important role.