Bryan Mesmer

Engineers commonly validate complex systems by considering arguments, pro and con, that a system possesses (or will possess) desired properties, like safety or high-quality performance. This argumentation is often conducted in service of a decision, like passing a critical design review, or deploying a system for a given task. It can question facts, methods, and inferences, including generalizations from test cases, analogies between application tasks, and expert opinion. However, very few formalisms support quantitative reasoning about system properties in the presence of such arguments. This paper develops a representation for stating conflicting arguments and evaluating such structures into probabilistic assertions about claims. We employ Toulmin argumentation models as a base; an informal representation of rhetorical argument that connects premises through warrants (justifications) to claims. We provide a novel, probabilistic interpretation of warrants as methods for making defeasible inferences by separating the relevance (or aptness) of the method from the likelihood of its conclusion. We use this interpretation to formally integrate Toulmin models into Influence diagrams. We illustrate the value of this integration for systems validation by employing the machinery of decision analysis to examine a decision in the presence of engineering arguments. We consider a choice between two rover wheel designs given a simulation package that affirms one option, and an expert who casts doubt on the validity of the simulation. Then, we determine the expected value of conducting a physical test to resolve a key uncertainty in the presence of these conflicting arguments.

The emergence and adoption of Model‐Based Systems Engineering (MBSE) is driven in part by claimed benefits over traditional Document‐Based Systems Engineering. Despite high enthusiasm for MBSE, few studies have evaluated the value of MBSE implementation or the needs in MBSE implementation across various system types and industry sectors, which may require differing drivers of implementation and analyses of added value. This study investigates the perceptions of MBSE across sectors including Commercial, Contractor , and Government , as well as across system types including Astronautical, Aeronautical , and Infrastructure , to determine the perceived drivers and considerations of MBSE implementation across different industries. From 60 academic sources, this study collected and coded over 2900 claims about MBSE using 68 MBSE attributes. Our study's findings determine that the considerations for MBSE implementation likely vary by both sector and by system. Perceptions of MBSE, regardless of application grouping, were largely positive—negative attributes often captured unfavorable perceptions of implementation and tooling immaturity. The findings of this study suggest that practitioners may need to tailor the implementation of MBSE partly to their organizational priorities and needs, and to be cautious of relying only on broadly reported benefits of MBSE.

The decisions that direct system development are based on assessments of a system, referred to as technical measures. Systems engineers select the set of technical measures for each system, informed by previous systems, guidebooks, and systems engineering research. If guidance for selecting technical measures is contradictory or does not ensure that a satisfactory set is selected, then the selected set of technical measures may not facilitate directing a system development as desired. Within large‐scale complex engineered systems development, a technical measure set provides clear targets for a system and can be used to coordinate and integrate efforts by many individuals and teams. This systematic literature review of 89 academic and industry developed engineering literature sources examines over 2500 guidance statements for technical measure selection. Most guidance provides examples or abstract qualities that technical measures should exhibit. Such guidance can result in multiple sets of technical measures that each drive toward different designs. The guidance identifies origins for technical measure selection, such as from stakeholders, but lacks detail on how to elicit a set of technical measures from stakeholders. This research's systematic literature review audits the consistency, contradictions, and omissions in current guidance for selecting technical measures. From these findings, we seek to establish evidence‐based research needs in technical measure selection.

Systems often inspire the development of new models to manage size and complexity challenges in system development. Engineers should be aware of overlap in the models used for systems engineering of systems as it would be inefficient to reiterate overlapping tasks. The objective of this paper is to identify consistencies between two models that appear to have overlapping tasks: Goal-Function Trees (GFTs) and value models. Both models are perceived to represent information about the system and stakeholder preferences, but have historically used differing processes and language. An analysis of the functional use and terminology for the two models reveals consistencies. The two models are compared in this paper using content analysis of systems engineering academic literature. It is seen that physics-based system representation and stakeholder preference incorporation are often common to both models. A coffee maker is used to illustrate the potential consistencies and overlapping tasks between the models. Extending this simple example, the applicability of the models to space systems is discussed. The findings of this article can be used by systems engineers to improve their efficiency when applying both GFTs and value models.

The objective of this research is to develop a methodology for an arts‐based intervention that can effectively promote employee voice. An arts‐based intervention was developed to promote employee voice, supported by literature and expert knowledge. This intervention was applied at the 2019 NASA Cost and Schedule Symposium. Participants indicated that the intervention was effective in eliciting “hidden truths.” Several topics were revealed that participants felt uncomfortable discussing in the workplace. The intervention methodology is evaluated, considering limitations and future changes.

Although Model‐Based Systems Engineering (MBSE) is quickly becoming adopted in Systems Engineering (SE), there have not been many studies evaluating literature to determine the perceived value of implementing MBSE. This research first identifies and discusses previous studies on the justification or rejection of MBSE. This study investigates how the SE community perceives the value of MBSE by coding and analyzing positive and negative descriptions of MBSE; perceived benefits and drawbacks of implementing MBSE; and the evidence and metrics used to substantiate and measure each claim about MBSE. From 60 academic sources, this study collected and coded over 2900 claims on MBSE. Our findings determine the most positive attributes of MBSE to be Verification & Validation Capability, Consistency, Reasoning , and Risk & Error Manageability , while the most negative attributes were Approach Understandability, Acceptability, Familiarity , and Approach Complexity . The most‐stated benefits were Reduced Time, Better Communication/Information Sharing, Reduced Costs , and Better Analysis Capability . The most claimed drawbacks were Increased Costs, Increased Time, Increased Effort , and Worsened Capability . A large share of claims (47%) about MBSE was based on author opinions. Most claims (86%) were not substantiated by a metric.

Background: Communication is the backbone of effective collaboration, enabling project success; yet, engineering projects often fail due to poor communication. Specifically, engineering teams may benefit from reflexivity interventions to improve decision making, problem solving, innovation, and performance. In this study, we focus on team reflexivity in direct application to engineering project management to identify reflexivity processes that facilitate effective communication. Literature review: Although research has shown that team reflexivity interventions—elicited through communication—can improve team interaction and performance, little empirical evidence exists into the temporal dimensions across the action and transition phases of team reflexivity processes. Research questions: 1. How is team reflexivity expressed through text-based communication? 2. How and when do team members shift between reflexivity processes over time, especially across and between transition and action phases? Research methodology: We analyzed collaborative activity among 62 four-person teams in a computer-simulated microworld across two scenarios. The reflexivity processes exhibited during interaction were identified and analyzed using statistical and content analysis. Results and discussion: Analyses indicated that team reflection promoted discussions about key issues, facilitated frequent process shifts among transition and action phases, and resulted in overall better performance. Conclusions: Our findings demonstrate the importance of team reflexivity interventions in engineering project teams to strategically guide members to improve planning, responding, and quality of attention devoted to long-term outcomes. Furthermore, our findings demonstrate the importance of deep structured team reflexivity through process shifts to help members understand strategies and goals, and develop shared objectives in complex environments.

View Video Presentation: https://doi.org/10.2514/6.2022-2136.vid Often characterized as nonfunctional system characteristics, system engineering ‘ilities’ are increasing in popularity as system complexity increases. Affordability, even without a widely accepted definition, is becoming a more apparent ‘ility’ within the systems engineering community. Varying definitions within the community for affordability can lead to a disjointed approach to developing more affordable systems. The aim of the current study is to discern how individuals approach the concept of affordability and discover the homogeneity of improving affordability approaches within a single organization. To examine this, employees at the National Aeronautics and Space Administration Marshall Space Flight Center were asked to complete a multipart survey including questions regarding their approach to affordability. While academic sources often invoke affordability as a more holistic term, the results (N=173) indicate that the majority of participants associate affordability with a financial indicator such as budget or cost reasonableness. The purpose of this paper is to present the results and expand on the study’s findings of individual participants’ affordability definitions from the survey conducted.

Background: Communication is fundamental to the success of engineered systems, enabling interactions between the system's stakeholders. Systems engineering, an integrative discipline on which the contributions of many disciplines are evaluated against each other, may particularly benefit from research in communication methods. Specifically, storytelling may be beneficial to engineers because it enables sense-making. Research into storytelling is conducted to identify storytelling metrics that could be useful in engineering communication, specifically engineering case studies. Literature review: Although storytelling has been identified in past research as possibly useful to Model-Based Systems Engineering (MBSE) and software requirement writing, a rigorous study of the use of storytelling elements in systems engineering communication has not been performed. Research question: How are storytelling elements currently being applied in engineering case studies? Research methodology: Twelve interdisciplinary metrics from storytelling, content analysis, and engineering are identified from the literature and used to characterize a collection of 48 NASA case studies. The values of the metrics for each case study are determined and analyzed using statistical and content analyses. Results and discussion: Analysis of the 12 metrics indicates that the case study design region with a historical backstory structure, climactic plot structure, and early points of attack is most frequently used by designers. Conclusions: The analysis indicates that certain storytelling elements applied in engineering case studies are used more frequently. Further work is needed to leverage the metrics as design variables in engineering case study writing.