Thomas Stefani

The European Union Aviation Safety Agency (EASA) is developing guidelines to certify AI-based systems in aviation with learning assurance as a key framework. Central to the learning assurance are the definitions of a Concept of Operations, an Operational Domain, and an AI/ML constituent Operational Design Domain (ODD). However, because no further guidance on these concepts is provided to developers, this work introduces a framework for defining them. For the concepts of the Operational Domain of the overall system and the AI/ML constituent ODD, a tabular definition language is introduced. Furthermore, processes are introduced to define the different necessary artifacts. During the specification process for the AI/ML constituent ODD, existing steps were identified and consolidated, including the identification of domain-specific concepts for the AI/ML constituent. To validate the framework, it was applied to the pyCASX system, which employs neural-network-based compression. For this use case, the framework produced an AI/ML constituent ODD with finer detail than other ODDs defined for the same airborne collision avoidance use case. Thus, the proposed novel framework is an important step toward a holistic approach aligned with EASA’s guidelines.

The European Union Aviation Safety Agency (EASA) is developing guidelines to certify AI-based systems in aviation with learning assurance as a key framework. Central to the learning assurance are the definitions of a Concept of Operations, an Operational Domain, and an AI/ML constituent Operational Design Domain (ODD). However, since no further guidance for these concepts is provided to developers, this work introduces a methodology for their definition. Concerning the concepts of the Operational Domain of the overall system and the AI/ML constituent ODD, a tabular definition language for both is introduced. Furthermore, processes are introduced to define the different necessary artifacts. For the specification process of the AI/ML constituent ODD, different preexisting steps were identified and combined, such as the identification of domain-specific concepts for the AI/ML constituent. To validate the methodology, it was applied to the pyCASX system that utilizes neural network-based compression. For the use case, the methodology proved it was able to produce an AI/ML constituent ODD of finer detail compared to other ODDs defined for the same airborne collision avoidance use case. Thus, the proposed novel framework is an important step toward a holistic framework following EASA’s guidelines.

A continuous increase in artificial intelligence (AI)-based functions can be expected for future aviation systems, posing significant challenges to traditional development processes. Established systems engineering frameworks, such as the V-model, are not adequately addressing the novel challenges associated with AI-based systems. Consequently, the European Union Aviation Safety Agency (EASA) introduced the W-shaped process, an advancement of the V-model, to set a regulatory framework for the novel challenges of AI Engineering. In contrast, the agile Development Operations (DevOps) approach, widely adopted in software development, promotes a never-ending iterative development process. This article proposes a novel concept that integrates aspects of DevOps into the W-shaped process to create an AI Engineering framework suitable for aviation-specific applications. Furthermore, it builds upon proven ideas and methods using AI Engineering efforts from other domains. The proposed extension of the W-shaped process, compatible with ongoing standardizations from the G34/WG-114 Standardization Working Group, a joint effort between EUROCAE and SAE, addresses the need for a rigorous development process for AI-based systems while acknowledging its limitations and potential for future advancements. The proposed framework allows for a re-evaluation of the AI/ML constituent based on operational information, enabling improvements of the system’s capabilities with each iteration.

A continuous increase of Artificial Intelligence (AI) based functions can be expected for future aviation systems, posing significant challenges to traditional development processes. Established systems engineering frameworks, such as the V-model, are not adequately addressing the novel challenges associated with AI-based systems. Consequently, the European Union Aviation Safety Agency (EASA) introduced the W-shaped process as an advancement of the V-model to set a regulatory framework for the novel challenges of AI Engineering. In contrast, the agile Development Operations (DevOps) approach, widely adopted in software development, promotes a never-ending iterative development process. This article proposes a novel concept that integrates aspects of DevOps into the W-shaped process to create an AI Engineering framework suitable for aviation-specific applications. Furthermore, it builds upon proven ideas and methods using AI Engineering efforts from other domains. The proposed extension of the W-shaped process, compatible with ongoing standardizations from the G34/WG-114 Standardization Working Group, a joint effort between EUROCAE and SAE, addresses the need for a rigorous development process for AI-based systems while acknowledging its limitations and potential for future advancements. The proposed framework allows for a re-evaluation of the AI/ML constituent based on information from operations, enabling improvement of the system’s capabilities in each iteration.

Applications based on artificial intelligence (AI) promise benefits, ranging from improved performance to increased capabilities in many industries. In the aviation domain, one example is the new Airborne Collision Avoidance System (ACAS X). The current investigation aims at combining ACAS X and AI to maintain its performance while decreasing the memory footprint. However, the anticipation of AI being increasingly used confronts regulators with challenges in terms of safety assurance and certification. Consequently, the European Union Aviation Safety Agency (EASA) published a concept paper for machine learning applications in aviation. Both, the Concept of Operation (ConOps) in combination with an Operational Design Domain (ODD), are listed as objectives to be met for the safety analysis. From a developer’s perspective, this raises questions on how to effectively derive the ODD from ConOps and test the given system based on the ODD description. Based on an exemplary use case of a Near Mid-Air Collision avoidance between two aircraft through the advisories of ACAS X, a highly automated framework for generating and testing synthetic data is proposed. Using this framework, 1800 Near Mid-Air Collision scenario files are created and automatically executed in the simulation environment FlightGear. Scenario-based testing is used for the logging of ACAS X advisory data and evaluating it against predefined requirements. By this approach, an efficient way of verifying system requirements and conducting automated testing based on the ODD definition is demonstrated. Throughout this process, Model-Based Systems Engineering (MBSE) is used to reduce and manage complexity. The framework in this paper enables a systematic and highly automated approach for scenario generation based on the ODD.