Cor Steging
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Cor Steging has a total of 4 co-authors. Here are the top co-authors by number of shared publications:
- Bart Verheij
University of Groningen
2 shared publications - Silja Renooij
Utrecht University
2 shared publications - Ludi van Leeuwen
Bernoulli Institute for Mathematics, Computer Science and Artificial Intelligence, University of Groningen
1 shared publication - Tadeusz Zbiegień
Department of Legal Theory, Jagiellonian University, Kraków, Poland
1 shared publication
Latest Publications
- Open Access
- 4 March 2026
- Published by Springer Nature in The Review of Socionetwork Strategies
- Vol. 20 (1) , 255-279
Investigating Expert-Based Prompt Engineering for Legal Entailment Tasks
Legal reasoning is complex and multi-faceted, requiring a broad set of skills. By employing domain knowledge from legal experts, we design five elements that can be included in prompts for large language models that could aid in legal reasoning tasks. We use additional legal guidelines, 1-shot prompting, dictionary definitions, knowledge representations of legal articles, and IRAC-style prompting. We investigate the effect of each prompt element on the model’s performance on a legal entailment task. Certain prompt elements can improve performance, depending on the context and the model. For the smaller models, increasing the number of prompt elements improves performance on average. For any particular combination of model and sub-task, only using a subset of the prompt elements seems to work best. For the most advanced reasoning model we evaluate, using a selection of prompt elements increases average performance across all evaluated sub-tasks. Results indicate that the problem space of the legal entailment task may be too large for a single model and prompt. In future research, we therefore aim to investigate the capabilities of an ensemble of specialized models.
- 4 March 2026
- Published by Springer Nature in The Review of Socionetwork Strategies
- Vol. 20 (1) , 255-279
Open AccessInvestigating Expert-Based Prompt Engineering for Legal Entailment Tasks
Legal reasoning is complex and multi-faceted, requiring a broad set of skills. By employing domain knowledge from legal experts, we design five elements that can be included in prompts for large language models that could aid in legal reasoning tasks. We use additional legal guidelines, 1-shot prompting, dictionary definitions, knowledge representations of legal articles, and IRAC-style prompting. We investigate the effect of each prompt element on the model’s performance on a legal entailment task. Certain prompt elements can improve performance, depending on the context and the model. For the smaller models, increasing the number of prompt elements improves performance on average. For any particular combination of model and sub-task, only using a subset of the prompt elements seems to work best. For the most advanced reasoning model we evaluate, using a selection of prompt elements increases average performance across all evaluated sub-tasks. Results indicate that the problem space of the legal entailment task may be too large for a single model and prompt. In future research, we therefore aim to investigate the capabilities of an ensemble of specialized models.
- Open Access
- 16 June 2025
- Published by Association for Computing Machinery (ACM)
- p. 455-459
Parameterized Argumentation-based Reasoning Tasks for Benchmarking Generative Language Models
- 16 June 2025
- Published by Association for Computing Machinery (ACM)
- p. 455-459
Open AccessParameterized Argumentation-based Reasoning Tasks for Benchmarking Generative Language Models
- Open Access
Parameterized Argumentation-based Reasoning Tasks for Benchmarking Generative Language Models
Generative large language models as tools in the legal domain have the potential to improve the justice system. However, the reasoning behavior of current generative models is brittle and poorly understood, hence cannot be responsibly applied in the domains of law and evidence. In this paper, we introduce an approach for creating benchmarks that can be used to evaluate the reasoning capabilities of generative language models. These benchmarks are dynamically varied, scalable in their complexity, and have formally unambiguous interpretations. In this study, we illustrate the approach on the basis of witness testimony, focusing on the underlying argument attack structure. We dynamically generate both linear and non-linear argument attack graphs of varying complexity and translate these into reasoning puzzles about witness testimony expressed in natural language. We show that state-of-the-art large language models often fail in these reasoning puzzles, already at low complexity. Obvious mistakes are made by the models, and their inconsistent performance indicates that their reasoning capabilities are brittle. Furthermore, at higher complexity, even state-of-the-art models specifically presented for reasoning capabilities make mistakes. We show the viability of using a parametrized benchmark with varying complexity to evaluate the reasoning capabilities of generative language models. As such, the findings contribute to a better understanding of the limitations of the reasoning capabilities of generative models, which is essential when designing responsible AI systems in the legal domain.
Open AccessParameterized Argumentation-based Reasoning Tasks for Benchmarking Generative Language Models
Generative large language models as tools in the legal domain have the potential to improve the justice system. However, the reasoning behavior of current generative models is brittle and poorly understood, hence cannot be responsibly applied in the domains of law and evidence. In this paper, we introduce an approach for creating benchmarks that can be used to evaluate the reasoning capabilities of generative language models. These benchmarks are dynamically varied, scalable in their complexity, and have formally unambiguous interpretations. In this study, we illustrate the approach on the basis of witness testimony, focusing on the underlying argument attack structure. We dynamically generate both linear and non-linear argument attack graphs of varying complexity and translate these into reasoning puzzles about witness testimony expressed in natural language. We show that state-of-the-art large language models often fail in these reasoning puzzles, already at low complexity. Obvious mistakes are made by the models, and their inconsistent performance indicates that their reasoning capabilities are brittle. Furthermore, at higher complexity, even state-of-the-art models specifically presented for reasoning capabilities make mistakes. We show the viability of using a parametrized benchmark with varying complexity to evaluate the reasoning capabilities of generative language models. As such, the findings contribute to a better understanding of the limitations of the reasoning capabilities of generative models, which is essential when designing responsible AI systems in the legal domain.
- 10 September 2024
- Published by University of Groningen Press
Designing responsible artificial intelligence
- 10 September 2024
- Published by University of Groningen Press
Designing responsible artificial intelligence