Towards LLM-Assisted Business Process Modeling in an Industrial Modeling Tool: An Experience Report
Open Access
- 3 October 2026
- conference paper
- Published by Association for Computing Machinery (ACM)
- p. 188-199
Abstract
Business Process Modeling (BPM) is a key and complex activity in Software and Systems Engineering. As a consequence, any relevant assistance integrated into the BPM tooling can be highly beneficial. Historically, Natural Language Processing (NLP) techniques have already been used in this context. However, the recent developments around Large Language Models (LLMs) offer promising new perspectives. In this paper, we report on our practical experience of integrating state-of-the-art LLMs within the industrial modeling tool Modelio. In particular, we experiment with the automated generation of BPMN models from natural-language instructions. To this end, we consider two different solutions: 1) The direct generation of BPMN creation scripts that call the modeling tool API; 2) The generation of intermediate representations that use a BPMN helper library. We first validate our approach with 7 industrial partners on 24 scenarios from the MATISSE collaborative project. In addition, we evaluate this approach with 3 different LLMs across 55 BPM scenarios from an open dataset. We demonstrate that using an intermediate representation (expressed in a compact DSL) globally improves performance, especially in more complex cases. We also share practical lessons learned and guidelines for the potential adoption of our approach by practitioners and researchers.Funding Information
- Chips Joint Undertaking (MATISSE project) (101140216)
- French Agence Nationale de la Recherche Technologique (Cifre PhD grant) (2025/1657)
This publication has 19 references indexed in Scilit:
- Evaluating large language models on business process modeling: framework, benchmark, and self-improvement analysisSoftware and Systems Modeling, 2025
- Process Modeler vs. Chatbot: Is Generative AI Taking over Process Modeling?2025
- On the use of large language models in model-driven engineeringSoftware and Systems Modeling, 2025
- A Universal Prompting Strategy for Extracting Process Model Information from Natural Language Text Using Large Language Models2024
- Process Modeling with Large Language Models2024
- Large Language Models for Business Process Management: Opportunities and Challenges2023
- On the assessment of generative AI in modeling tasks: an experience report with ChatGPT and UMLSoftware and Systems Modeling, 2023
- Extracting Declarative Process Models from Natural Language2019
- A Comprehensive Investigation of BPMN Models Generation from Textual Requirements—Techniques, Tools and Trends2018
- Process Model Generation from Natural Language Text2011