Structured validation of AI-based systems by virtual testing in simulated test scenarios
Open Access
- 14 February 2023
- journal article
- research article
- Published by Springer Nature in Applied Intelligence
- Vol. 53 (15) , 18910-18924
Abstract
The growing relevance of artificial intelligence (AI) for technical systems offers significant potential for the realization and operation of autonomous systems in complex and potentially unknown environments. However, unlike classical solution approaches, the functionality of an AI system cannot be verified analytically, which is why data-driven approaches such as scenario-based testing are used. With the increasing complexity of the required functionality of the AI-based system, the quantity, and quality of the data needed for development and validation also increase. To meet this demand, data generated synthetically using simulation is increasingly being used. Compared to the acquisition of real-world reference data, simulation offers the major advantage that it can be configured to test specific scenarios of interest. This paper presents an architecture for the systematic generation of virtual test scenarios to establish synthetically generated test data as an integral part of the development and validation process for AI systems. Key aspects of this architecture are the consistent use of digital twins as virtual 1-to-1 replicas and a simulation infrastructure that enables the generation of training and validation data for AI-based systems in appropriate quantity, quality, and time. In particular, this paper focuses on the application of the architecture in the context of two use cases from different application domains.Keywords
Funding Information
- Bundesministerium für Wirtschaft und Technologie (50RA1934)
- RWTH Aachen University
This publication has 12 references indexed in Scilit:
- Explainable digital forensics AI: Towards mitigating distrust in AI-based digital forensics analysis using interpretable modelsForensic Science International: Digital Investigation, 2022
- Modeling Operational Scenarios for Simulation-based Validation of Technical Systems2022
- Formal Scenario-driven Logical Spaces for Randomized Synthetic Data Generation2022
- Generation of Virtual Test Scenarios for Training and Validation of AI-based Systems2021
- What is a Digital Twin – A Mediation Approach2021
- The Dataset Nutrition Label2020
- VerifAI: A Toolkit for the Formal Design and Analysis of Artificial Intelligence-Based Systems2019
- Scenic: a language for scenario specification and scene generation2019
- Interactive Analysis and Visualization of Digital Twins in High-Dimensional State Spaces2018
- Control by 3D Simulation – A New eRobotics Approach to Control Design in AutomationLecture Notes in Computer Science, 2012