Marcus Kessel
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Research Subjects
Research Interests
Marcus Kessel has a total of 9 co-authors. Here are the top co-authors by number of shared publications:
- Colin Atkinson
University of Mannheim
6 shared publications - Alexander Berndt
Heidelberg University, Heidelberg, Germany
2 shared publications - Rainer Gemulla
University of Mannheim, Mannheim, Germany
2 shared publications - Sebastian Baltes
Heidelberg University, Heidelberg, Germany
2 shared publications - Thomas Bach
SAP, Walldorf, Germany
2 shared publications Colin Atkinson
1 shared publication
Latest Publications
- Open Access
- 5 July 2026
- Published by Association for Computing Machinery (ACM)
- p. 1366-1370
Treating Run-time Execution History as a First-Class Citizen: Co-Versioning Run-time Behavior alongside Code
Behavioral Co-Versioning remains absent from mainstream practice: while developers routinely version source code with Git, they rarely persist and query how run-time behavior evolves across revisions. This paper argues that this mismatch contributes to a blind spot in software evolution analysis and CI, where rich execution information is discarded and typically reduced to pass/fail outcomes — despite partial test oracles, flakiness, and silent output or performance drift. We propose Behavioral Co-Versioning, a paradigm that couples the Git history with a Behavioral Archive: an append-only, queryable store of selected run-time observations (e.g., method I/O and performance signals) collected during test runs and keyed by commit and test context. This enables semantic diffing, behavior-aware regression localization, and retrospective auditing by querying historical executions, complementing proactive, signal-specific monitoring tools. We first outline a minimal data model and change diagnostics based on code/test/behavior fingerprints, and then demonstrate feasibility with a laptop-scale prototype that replays historical commits of a Python project, archives run-time observations in a local Parquet-backed store, and detects behavioral changes not apparent from textual diffs.
- 5 July 2026
- Published by Association for Computing Machinery (ACM)
- p. 1366-1370
Open AccessTreating Run-time Execution History as a First-Class Citizen: Co-Versioning Run-time Behavior alongside Code
Behavioral Co-Versioning remains absent from mainstream practice: while developers routinely version source code with Git, they rarely persist and query how run-time behavior evolves across revisions. This paper argues that this mismatch contributes to a blind spot in software evolution analysis and CI, where rich execution information is discarded and typically reduced to pass/fail outcomes — despite partial test oracles, flakiness, and silent output or performance drift. We propose Behavioral Co-Versioning, a paradigm that couples the Git history with a Behavioral Archive: an append-only, queryable store of selected run-time observations (e.g., method I/O and performance signals) collected during test runs and keyed by commit and test context. This enables semantic diffing, behavior-aware regression localization, and retrospective auditing by querying historical executions, complementing proactive, signal-specific monitoring tools. We first outline a minimal data model and change diagnostics based on code/test/behavior fingerprints, and then demonstrate feasibility with a laptop-scale prototype that replays historical commits of a Python project, archives run-time observations in a local Parquet-backed store, and detects behavioral changes not apparent from textual diffs.
- Open Access
- 12 April 2026
- Published by Association for Computing Machinery (ACM)
- p. 819-830
On the Flakiness of LLM-Generated Tests for Industrial and Open-Source Database Management Systems
- 12 April 2026
- Published by Association for Computing Machinery (ACM)
- p. 819-830
Open AccessOn the Flakiness of LLM-Generated Tests for Industrial and Open-Source Database Management Systems
- 17 March 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 709-713
Towards Observation Lakehouses: Living, Interactive Archives of Software Behavior
- 17 March 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 709-713
Towards Observation Lakehouses: Living, Interactive Archives of Software Behavior
- Open Access
Can We Classify Flaky Tests Using Only Test Code? An LLM-Based Empirical Study
Flaky tests yield inconsistent results when they are repeatedly executed on the same code revision. They interfere with automated quality assurance of code changes and hinder efficient software testing. Previous work evaluated approaches to train machine learning models to classify flaky tests based on identifiers in the test code. However, the resulting classifiers have been shown to lack generalizability, hindering their applicability in practical environments. Recently, pre-trained Large Language Models (LLMs) have shown the capability to generalize across various tasks. Thus, they represent a promising approach to address the generalizability problem of previous approaches. In this study, we evaluated three LLMs (two general-purpose models, one code-specific model) using three prompting techniques on two benchmark datasets from prior studies on flaky test classification. Furthermore, we manually investigated 50 samples from the given datasets to determine whether classifying flaky tests based only on test code is feasible for humans. Our findings indicate that LLMs struggle to classify flaky tests given only the test code. The results of our best prompt-model combination were only marginally better than random guessing. In our manual analysis, we found that the test code does not necessarily contain sufficient information for a flakiness classification. Our findings motivate future work to evaluate LLMs for flakiness classification with additional context, for example, using retrieval-augmented generation or agentic AI.
Open AccessCan We Classify Flaky Tests Using Only Test Code? An LLM-Based Empirical Study
Flaky tests yield inconsistent results when they are repeatedly executed on the same code revision. They interfere with automated quality assurance of code changes and hinder efficient software testing. Previous work evaluated approaches to train machine learning models to classify flaky tests based on identifiers in the test code. However, the resulting classifiers have been shown to lack generalizability, hindering their applicability in practical environments. Recently, pre-trained Large Language Models (LLMs) have shown the capability to generalize across various tasks. Thus, they represent a promising approach to address the generalizability problem of previous approaches. In this study, we evaluated three LLMs (two general-purpose models, one code-specific model) using three prompting techniques on two benchmark datasets from prior studies on flaky test classification. Furthermore, we manually investigated 50 samples from the given datasets to determine whether classifying flaky tests based only on test code is feasible for humans. Our findings indicate that LLMs struggle to classify flaky tests given only the test code. The results of our best prompt-model combination were only marginally better than random guessing. In our manual analysis, we found that the test code does not necessarily contain sufficient information for a flakiness classification. Our findings motivate future work to evaluate LLMs for flakiness classification with additional context, for example, using retrieval-augmented generation or agentic AI.
- Open Access
- 26 May 2025
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Software Engineering and Methodology
- Vol. 34 (5) , 1-17
Morescient GAI for Software Engineering
The ability of Generative AI (GAI) technology to automatically check, synthesize, and modify software engineering artifacts promises to revolutionize all aspects of software engineering. Using GAI for software engineering tasks is consequently one of the most rapidly expanding fields of software engineering research, with over a hundred LLM-based code models having been published since 2021. However, the overwhelming majority of existing code models share a major weakness—they are exclusively trained on the syntactic facet of software, significantly lowering their trustworthiness in tasks dependent on software semantics. To address this problem, a new class of “Morescient” GAI is needed that is “aware” of (i.e., trained on) both the semantic and static facets of software. This, in turn, will require a new generation of software observation platforms capable of generating large quantities of execution observations in a structured and readily analyzable way. In this article, we present a vision and roadmap for how such “Morescient” GAI models can be engineered, evolved, and disseminated according to the principles of open science.
- 26 May 2025
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Software Engineering and Methodology
- Vol. 34 (5) , 1-17
Open AccessMorescient GAI for Software Engineering
The ability of Generative AI (GAI) technology to automatically check, synthesize, and modify software engineering artifacts promises to revolutionize all aspects of software engineering. Using GAI for software engineering tasks is consequently one of the most rapidly expanding fields of software engineering research, with over a hundred LLM-based code models having been published since 2021. However, the overwhelming majority of existing code models share a major weakness—they are exclusively trained on the syntactic facet of software, significantly lowering their trustworthiness in tasks dependent on software semantics. To address this problem, a new class of “Morescient” GAI is needed that is “aware” of (i.e., trained on) both the semantic and static facets of software. This, in turn, will require a new generation of software observation platforms capable of generating large quantities of execution observations in a structured and readily analyzable way. In this article, we present a vision and roadmap for how such “Morescient” GAI models can be engineered, evolved, and disseminated according to the principles of open science.
- 27 September 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Software
- Vol. 42 (2) , 76-83
N-Version Assessment and Enhancement of Generative AI: Differential GAI
Generative AI (GAI) holds great potential to improve software engineering productivity, but its untrustworthy outputs, particularly in code synthesis, pose significant challenges. The need for extensive verification and validation (V&V) of GAI-generated artifacts may undermine the potential productivity gains. This paper proposes a way of mitigating these risks by exploiting GAI’s ability to generate multiple versions of code and tests to facilitate comparative analysis across versions. Rather than relying on the quality of a single test or code module, this “differential GAI” (D-GAI) approach promotes more reliable quality evaluation through version diversity. We introduce the Large-Scale Software Observatorium (LASSO), a platform that supports D-GAI by executing and analyzing large sets of code versions and tests. We discuss how LASSO enables rigorous evaluation of GAI-generated artifacts and propose its application in both software development and GAI research.
- 27 September 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Software
- Vol. 42 (2) , 76-83
N-Version Assessment and Enhancement of Generative AI: Differential GAI
Generative AI (GAI) holds great potential to improve software engineering productivity, but its untrustworthy outputs, particularly in code synthesis, pose significant challenges. The need for extensive verification and validation (V&V) of GAI-generated artifacts may undermine the potential productivity gains. This paper proposes a way of mitigating these risks by exploiting GAI’s ability to generate multiple versions of code and tests to facilitate comparative analysis across versions. Rather than relying on the quality of a single test or code module, this “differential GAI” (D-GAI) approach promotes more reliable quality evaluation through version diversity. We introduce the Large-Scale Software Observatorium (LASSO), a platform that supports D-GAI by executing and analyzing large sets of code versions and tests. We discuss how LASSO enables rigorous evaluation of GAI-generated artifacts and propose its application in both software development and GAI research.
- Open Access
- 6 May 2024
- Published by Elsevier in Journal of Systems and Software
- Vol. 215
Code search engines for the next generation
- 6 May 2024
- Published by Elsevier in Journal of Systems and Software
- Vol. 215
Open AccessCode search engines for the next generation
- Open Access
- 12 March 2024
- Published by Elsevier in Journal of Systems and Software
- Vol. 212
Promoting open science in test-driven software experiments
- 12 March 2024
- Published by Elsevier in Journal of Systems and Software
- Vol. 212
Open AccessPromoting open science in test-driven software experiments
- 15 December 2023
- Published by Springer Nature
- p. 158-170
Robustness Analysis of Machine Learning Models Using Domain-Specific Test Data Perturbation
- 15 December 2023
- Published by Springer Nature
- p. 158-170
Robustness Analysis of Machine Learning Models Using Domain-Specific Test Data Perturbation
- 11 August 2022
- Published by Elsevier in Journal of Systems and Software
- Vol. 193
Diversity-driven unit test generation
- 11 August 2022
- Published by Elsevier in Journal of Systems and Software
- Vol. 193
Diversity-driven unit test generation