Johann Schumann
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Research Subjects
Johann Schumann has a total of 35 co-authors. Here are the top co-authors by number of shared publications:
- Florian Holzapfel
Technical University of Munich
2 shared publications - Ole J. Mengshoel
Carnegie Mellon University
2 shared publications - Patrick Moosbrugger
Vienna University of Technology, Treitlstrasse 3, 1040 Vienna, Austria
2 shared publications - Chetan Kulkarni
ETH Zurich KBR, Inc NASA Ames Research Center Luleå University of Technology ETH Zurich
1 shared publication - Corey Ippolito
NASA Ames Research Center, Moffett Field, CA 94035, USA
1 shared publication - David H. Bushnell
NASA Ames Research Center, Moffett Field, USA
1 shared publication
Latest Publications
- Open Access
On the Practices of Autonomous Systems Development: Survey-based Empirical Findings
Autonomous systems have gained an important role in many industry domains and are beginning to change everyday life. However, due to dynamically emerging applications and often proprietary constraints, there is a lack of information about the practice of developing autonomous systems. This paper presents the first part of the longitudinal study focused on establishing state-of-the-practice, identifying and quantifying the challenges and benefits, identifying the processes and standards used, and exploring verification and validation (V&V) practices used for the development of autonomous systems. The results presented in this paper are based on data about software systems that have autonomous functionality and may employ model-based software engineering (MBSwE) and reuse. These data were collected using an anonymous online survey that was administered in 2019 and were provided by experts with experience in development of autonomous systems and /or the use of MBSwE. Our current work is focused on repeating the survey to collect more recent data and discover how the development of autonomous systems has evolved over time.
Open AccessOn the Practices of Autonomous Systems Development: Survey-based Empirical Findings
Autonomous systems have gained an important role in many industry domains and are beginning to change everyday life. However, due to dynamically emerging applications and often proprietary constraints, there is a lack of information about the practice of developing autonomous systems. This paper presents the first part of the longitudinal study focused on establishing state-of-the-practice, identifying and quantifying the challenges and benefits, identifying the processes and standards used, and exploring verification and validation (V&V) practices used for the development of autonomous systems. The results presented in this paper are based on data about software systems that have autonomous functionality and may employ model-based software engineering (MBSwE) and reuse. These data were collected using an anonymous online survey that was administered in 2019 and were provided by experts with experience in development of autonomous systems and /or the use of MBSwE. Our current work is focused on repeating the survey to collect more recent data and discover how the development of autonomous systems has evolved over time.
- Open Access
- 8 October 2024
- Published by PHM Society in International Journal of Prognostics and Health Management
- Vol. 15 (3)
Statistical Analysis and Runtime Monitoring for an AI-based Autonomous Centerline Tracking System
Autonomous Centerline Tracking (ACT) enables an unmanned aircraft to be guided down the center of the runway, using a camera-based Deep Neural Network (DNN). ACT is safety-critical. The EASA Guidelines for machine-learning based systems list numerous assurance objectives that must be met toward certification and V&V. We extend our statistical analysis framework SYSAI to support meeting assurance objectives for a complex safety-critical system with AI components, and describe a combination with a runtime monitoring architecture that also supports advanced risk mitigation to support safety assurance of a complex AI-based aerospace system.
- 8 October 2024
- Published by PHM Society in International Journal of Prognostics and Health Management
- Vol. 15 (3)
Open AccessStatistical Analysis and Runtime Monitoring for an AI-based Autonomous Centerline Tracking System
Autonomous Centerline Tracking (ACT) enables an unmanned aircraft to be guided down the center of the runway, using a camera-based Deep Neural Network (DNN). ACT is safety-critical. The EASA Guidelines for machine-learning based systems list numerous assurance objectives that must be met toward certification and V&V. We extend our statistical analysis framework SYSAI to support meeting assurance objectives for a complex safety-critical system with AI components, and describe a combination with a runtime monitoring architecture that also supports advanced risk mitigation to support safety assurance of a complex AI-based aerospace system.
- Open Access
- 21 March 2024
- Published by Springer Nature in Aerospace Systems
- Vol. 7 (2) , 209-225
Low-level memory and timing analysis of flight code for unmanned aerial systems
The development of flight software for Unmanned Aerial Systems (UAS) is challenging due to the absence of an established development process defined by aerospace certification authorities. This research paper outlines our methods and tools for analyzing flight-critical UAS control software on the target hardware. We present our toolchain and methodology for evaluating the flight control computer stack, runtime memory, and timing characteristics. Additionally, we compare the performance of the flight control computer under various hardware and cache settings to justify, which hardware features should be enabled. The tools and processes employed in this research are deployable to any other development environment and are not restricted to the specific target hardware used in this paper.
- 21 March 2024
- Published by Springer Nature in Aerospace Systems
- Vol. 7 (2) , 209-225
Open AccessLow-level memory and timing analysis of flight code for unmanned aerial systems
The development of flight software for Unmanned Aerial Systems (UAS) is challenging due to the absence of an established development process defined by aerospace certification authorities. This research paper outlines our methods and tools for analyzing flight-critical UAS control software on the target hardware. We present our toolchain and methodology for evaluating the flight control computer stack, runtime memory, and timing characteristics. Additionally, we compare the performance of the flight control computer under various hardware and cache settings to justify, which hardware features should be enabled. The tools and processes employed in this research are deployable to any other development environment and are not restricted to the specific target hardware used in this paper.
- Open Access
Runway Sign Classifier: A DAL C Certifiable Machine Learning System
In recent years, the remarkable progress of Machine Learning (ML) technologies within the domain of Artificial Intelligence (AI) systems has presented unprecedented opportunities for the aviation industry, paving the way for further advancements in automation, including the potential for single pilot or fully autonomous operation of large commercial airplanes. However, ML technology faces major incompatibilities with existing airborne certification standards, such as ML model traceability and explainability issues or the inadequacy of traditional coverage metrics. Certification of ML-based airborne systems using current standards is problematic due to these challenges. This paper presents a case study of an airborne system utilizing a Deep Neural Network (DNN) for airport sign detection and classification. Building upon our previous work, which demonstrates compliance with Design Assurance Level (DAL) D, we upgrade the system to meet the more stringent requirements of Design Assurance Level C. To achieve DAL C, we employ an established architectural mitigation technique involving two redundant and dissimilar Deep Neural Networks. The application of novel ML-specific data management techniques further enhances this approach. This work is intended to illustrate how the certification challenges of ML-based systems can be addressed for medium criticality airborne applications.
Open AccessRunway Sign Classifier: A DAL C Certifiable Machine Learning System
In recent years, the remarkable progress of Machine Learning (ML) technologies within the domain of Artificial Intelligence (AI) systems has presented unprecedented opportunities for the aviation industry, paving the way for further advancements in automation, including the potential for single pilot or fully autonomous operation of large commercial airplanes. However, ML technology faces major incompatibilities with existing airborne certification standards, such as ML model traceability and explainability issues or the inadequacy of traditional coverage metrics. Certification of ML-based airborne systems using current standards is problematic due to these challenges. This paper presents a case study of an airborne system utilizing a Deep Neural Network (DNN) for airport sign detection and classification. Building upon our previous work, which demonstrates compliance with Design Assurance Level (DAL) D, we upgrade the system to meet the more stringent requirements of Design Assurance Level C. To achieve DAL C, we employ an established architectural mitigation technique involving two redundant and dissimilar Deep Neural Networks. The application of novel ML-specific data management techniques further enhances this approach. This work is intended to illustrate how the certification challenges of ML-based systems can be addressed for medium criticality airborne applications.
- Open Access
- 2 October 2017
- Published by PHM Society in Annual Conference of the PHM Society
- Vol. 9 (1)
Unobtrusive Software and System Health Management with R2U2 on a parallel MIMD Coprocessor
Dynamic monitoring of software and system health of a complex cyber-physical system requires observers that continuously monitor variables of the embedded software in order to detect anomalies and reason about their root causes. There exists a variety of techniques for code instrumentation, but instrumentation might change runtime behavior and could require costly software re-certification. In this paper, we present R2U2/E, a novel realization of our real-time, Realizable, Responsive, and Unobtrusive Unit (R2U2). The R2U2/E observers are executed in parallel on a dedicated 16 or 64 core EPIPHANY co-processor, thereby avoiding additional computational overhead to the system under observation. A DMA-based shared memory access architecture allows R2U2/E to operate without any code instrumentation or program interference.
- 2 October 2017
- Published by PHM Society in Annual Conference of the PHM Society
- Vol. 9 (1)
Open AccessUnobtrusive Software and System Health Management with R2U2 on a parallel MIMD Coprocessor
Dynamic monitoring of software and system health of a complex cyber-physical system requires observers that continuously monitor variables of the embedded software in order to detect anomalies and reason about their root causes. There exists a variety of techniques for code instrumentation, but instrumentation might change runtime behavior and could require costly software re-certification. In this paper, we present R2U2/E, a novel realization of our real-time, Realizable, Responsive, and Unobtrusive Unit (R2U2). The R2U2/E observers are executed in parallel on a dedicated 16 or 64 core EPIPHANY co-processor, thereby avoiding additional computational overhead to the system under observation. A DMA-based shared memory access architecture allows R2U2/E to operate without any code instrumentation or program interference.
- Open Access
- 12 April 2017
- Published by Springer Nature in Formal Methods in System Design
- Vol. 51 (1) , 31-61
R2U2: monitoring and diagnosis of security threats for unmanned aerial systems
- 12 April 2017
- Published by Springer Nature in Formal Methods in System Design
- Vol. 51 (1) , 31-61
Open AccessR2U2: monitoring and diagnosis of security threats for unmanned aerial systems
- 20 September 2016
- Published by Springer Nature
- p. 504-509
Runtime Analysis with R2U2: A Tool Exhibition Report
- 20 September 2016
- Published by Springer Nature
- p. 504-509
Runtime Analysis with R2U2: A Tool Exhibition Report
- Open Access
- 15 November 2015
- Published by Springer Nature
- p. 233-249
R2U2: Monitoring and Diagnosis of Security Threats for Unmanned Aerial Systems
- 15 November 2015
- Published by Springer Nature
- p. 233-249
Open AccessR2U2: Monitoring and Diagnosis of Security Threats for Unmanned Aerial Systems
- Open Access
- 18 October 2015
- Published by PHM Society in Annual Conference of the PHM Society
- Vol. 7 (1)
Diagnostic Reasoning using Prognostic Information for Unmanned Aerial Systems
With increasing popularity of unmanned aircraft, continuous monitoring of their systems, software, and health status is becoming more and more important to ensure safe, correct, and efficient operation and fulfillment of missions. The paper presents integration of prognosis models and prognostic information with the R2U2 (REALIZABLE, RESPONSIVE, and UNOBTRUSIVE Unit) monitoring and diagnosis framework. This integration makes available statistically reliable health information predictions of the future at a much earlier time to enable autonomous decision making. The prognostic information can be used in the R2U2 model to improve diagnostic accuracy and enable decisions to be made at the present time to deal with events in the future. This will be an advancement over the current state of the art, where temporal logic observers can only do such valuation at the end of the time interval. Usefulness and effectiveness of this integrated diagnostics and prognostics framework was demonstrated using simulation experiments with the NASA Dragon Eye electric unmanned aircraft.
- 18 October 2015
- Published by PHM Society in Annual Conference of the PHM Society
- Vol. 7 (1)
Open AccessDiagnostic Reasoning using Prognostic Information for Unmanned Aerial Systems
With increasing popularity of unmanned aircraft, continuous monitoring of their systems, software, and health status is becoming more and more important to ensure safe, correct, and efficient operation and fulfillment of missions. The paper presents integration of prognosis models and prognostic information with the R2U2 (REALIZABLE, RESPONSIVE, and UNOBTRUSIVE Unit) monitoring and diagnosis framework. This integration makes available statistically reliable health information predictions of the future at a much earlier time to enable autonomous decision making. The prognostic information can be used in the R2U2 model to improve diagnostic accuracy and enable decisions to be made at the present time to deal with events in the future. This will be an advancement over the current state of the art, where temporal logic observers can only do such valuation at the end of the time interval. Usefulness and effectiveness of this integrated diagnostics and prognostics framework was demonstrated using simulation experiments with the NASA Dragon Eye electric unmanned aircraft.
- Open Access
- 29 September 2014
- Published by PHM Society in Annual Conference of the PHM Society
- Vol. 6 (1)
A Tool Chain for the V&V of NASA Cryogenic Fuel Loading Health Management
Complex machinery like spacecraft, aircraft, or chemical plants are equipped with fault detection and diagnosis systems. Due to their safety-critical nature, such diagnosis systems have to undergo rigorous Verification and Validation (V&V). In this paper, we present a tool suite to facilitate V&V of the deployed diagnostic system. The V&V relies on the paradigms of cross validation (to compare the diagnosis results of the deployed reasoner against those of other, more advanced reasoners), automatic fault scenario generation (to support extensive testing and coverage analysis), and parametric model analysis (to enrich test sets and for robustness and sensitivity analysis). We present the application of this tool architecture towards the V&V of the diagnosis system based on the TEAMS tool suite towards a subsystem in the NASA cryogenic fuel loading facility.
- 29 September 2014
- Published by PHM Society in Annual Conference of the PHM Society
- Vol. 6 (1)
Open AccessA Tool Chain for the V&V of NASA Cryogenic Fuel Loading Health Management
Complex machinery like spacecraft, aircraft, or chemical plants are equipped with fault detection and diagnosis systems. Due to their safety-critical nature, such diagnosis systems have to undergo rigorous Verification and Validation (V&V). In this paper, we present a tool suite to facilitate V&V of the deployed diagnostic system. The V&V relies on the paradigms of cross validation (to compare the diagnosis results of the deployed reasoner against those of other, more advanced reasoners), automatic fault scenario generation (to support extensive testing and coverage analysis), and parametric model analysis (to enrich test sets and for robustness and sensitivity analysis). We present the application of this tool architecture towards the V&V of the diagnosis system based on the TEAMS tool suite towards a subsystem in the NASA cryogenic fuel loading facility.