Annemae van de Hoef

System understanding is crucial for maintaining and reengineering software systems. This holds particularly for legacy systems, where information about the system is often not documented well and numerous changes over time have increased their complexity. These systems incur high maintenance costs, make it difficult to implement new functionalities, and pose challenges for reengineering activities such as modernizing parts of the system. Moreover, replacing these systems is often infeasible due to the risk of losing embedded business knowledge. Documenting the business logic embedded within such systems, for instance by means of models using Decision Model and Notation (DMN), could help address this challenge. However, the manual extraction of DMN models is difficult given the inherent complexity of these systems. In this regard, Large Language Models (LLMs) offer the potential to support the automation of this process by analyzing source code, leveraging contextual information, and performing without the need for extensive task-specific training data. In this study, we explore whether and how LLMs (GPT-4.1 and Gemini 2.5 Pro) can be applied for automated extraction of DMN models from source code. We applied the Design Science Research (DSR) methodology, supported by a theoretical literature review that identified requirements for LLM-based DMN extraction. In addition, we conducted experiments to determine prompts, model configurations, and the extent to which the extraction process could be automated. Our results indicate that LLM-based automated extraction of DMN models from source code can serve as an aid for understanding complex software systems.

Operational decisions significantly influence organizational performance and individual well-being. Decision mining offers a method to discover and analyze decision logic from decision logs, enhancing decision-making processes. However, evaluating the quality of decision discovery algorithms remains a challenge. While precision, fitness, generalization, and simplicity are well-established quality dimensions in process mining, their adaptation to the decision mining domain is underexplored. This study adapts these four dimensions to the necessary characteristics of decision models, providing a framework for evaluating decision discovery algorithms. Using a design science research approach, we develop tailored metrics and functions and demonstrate their application through a practical example of environmental permit management modeled in Decision Model and Notation (DMN). Precision measures how the discovered decision model reproduces the observed fact types and values from the decision log, detecting over-specification in the decision model. Fitness evaluates how completely the decision model covers the behavior in the decision log, identifying missing or under-specified elements in the decision model. Generalization assesses the model’s robustness to unseen decision cases by quantifying how well the discovered decision model performs beyond the training data. Simplicity captures the complexity in the discovered decision model in relation to a human actor-specified threshold. These insights guide decision model improvements, contributing to higher transparency, accountability, and fairness in operational decision-making processes. This research bridges a gap in the body of knowledge by providing a concrete methodology for evaluating decision discovery algorithms. The results support organizations in aligning decision models with regulatory requirements and public values, while also laying a foundation for future research.

The translation from laws and regulations into actionable business rules remains challenging due to the complexity of Dutch legal text. In addition, the (semi-)manual translation of law into business rules is both time-consuming and error-prone. To address these issues, this research explores the use of Natural Language Processing (NLP) to automatically extract legal decisions and represent them in a Decision Model and Notation (DMN) model. For this purpose, existing research was reviewed to define requirements, which formed the basis for the NLP prototype. The current prototype evaluates an existing approach and aims to process unstructured Dutch legal text. However, a theoretical extension is proposed to address the structural complexity of extracting a DMN model from structured Dutch legal texts. Therefore, future research should focus on implementing the proposed approach and validating it in collaboration with legal analysts to extract a DMN model from structured Dutch legal texts.