LLMs that Understand Processes: Instruction-tuning for Semantics-Aware Process Mining
Preprint
Abstract
Process mining is increasingly using textual information associated with events to tackle tasks such as anomaly detection and process discovery. Such semantics-aware process mining focuses on what behavior should be possible in a process (i.e., expectations), thus providing an important complement to traditional, frequency-based techniques that focus on recorded behavior (i.e., reality). Large Language Models (LLMs) provide a powerful means for tackling semantics-aware tasks. However, the best performance is so far achieved through task-specific fine-tuning, which is computationally intensive and results in models that can only handle one specific task. To overcome this lack of generalization, we use this paper to investigate the potential of instruction-tuning for semantics-aware process mining. The idea of instruction-tuning here is to expose an LLM to prompt-answer pairs for different tasks, e.g., anomaly detection and next-activity prediction, making it more familiar with process mining, thus allowing it to also perform better at unseen tasks, such as process discovery. Our findings demonstrate a varied impact of instruction-tuning: while performance considerably improved on process discovery and prediction tasks, it varies across models on anomaly detection tasks, highlighting that the selection of tasks for instruction-tuning is critical to achieving desired outcomes.Keywords
All Related Versions
This publication has 16 references indexed in Scilit:
- On the potential of large language models to solve semantics-aware process mining tasksProcess Science, 2025
- PM-LLM-Benchmark: Evaluating Large Language Models on Process Mining Tasks2025
- Bridging Domain Knowledge and Process Discovery Using Large Language Models2025
- LUPIN: A LLM Approach for Activity Suffix Prediction in Business Process Event Logs2024
- Explanatory Capabilities of Large Language Models in Prescriptive Process Monitoring2024
- xSemAD: Explainable Semantic Anomaly Detection in Event Logs Using Sequence-to-Sequence Models2024
- Mapping the Landscape: Exploring Large Language Model Applications in Business Process Management2024
- Abstractions, Scenarios, and Prompt Definitions for Process Mining with LLMs: A Case Study2024
- Does This Make Sense? Machine Learning-Based Detection of Semantic Anomalies in Business Processes2023
- Natural language-based detection of semantic execution anomalies in event logsInformation Systems, 2021