MCMarcus Ciolkowski

Marcus Ciolkowski

*This is an algorithmically generated author record based on publicly available information.

Marcus Ciolkowski has a total of 72 co-authors. Here are the top co-authors by number of shared publications:


Latest Publications

  • Open Access

    DeepQuali: Initial results of a study on the use of large language models for assessing the quality of user stories

    by Adam Trendowicz, Daniel Seifert, Andreas Jedlitschka, Marcus Ciolkowski, Anton Strahilov

    Generative artificial intelligence (GAI), specifically large language models (LLMs), are increasingly used in software engineering, mainly for coding tasks. However, requirements engineering - particularly requirements validation - has seen limited application of GAI. The current focus of using GAI for requirements is on eliciting, transforming, and classifying requirements, not on quality assessment. We propose and evaluate the LLM-based (GPT-4o) approach "DeepQuali", for assessing and improving requirements quality in agile software development. We applied it to projects in two small companies, where we compared LLM-based quality assessments with expert judgments. Experts also participated in walkthroughs of the solution, provided feedback, and rated their acceptance of the approach. Experts largely agreed with the LLM's quality assessments, especially regarding overall ratings and explanations. However, they did not always agree with the other experts on detailed ratings, suggesting that expertise and experience may influence judgments. Experts recognized the usefulness of the approach but criticized the lack of integration into their workflow. LLMs show potential in supporting software engineers with the quality assessment and improvement of requirements. The explicit use of quality models and explanatory feedback increases acceptance.

  • Open Access

    The Right Amount of Technical Debt in an Agile Context

    by Marcus Ciolkowski, Philipp Diebold, Andrea Janes, Valentina Lenarduzzi

    Agile and technical debt management should have a symbiotic relationship, as technical debt was conceived as a metaphor (or tool) to balance the benefits of taking shortcuts for early release and user feedback with the responsibility of ‘repairing’ the effects of these trade-offs. Agile processes provide the necessary flexibility to achieve this balance. However, in reality, feature greed often takes over, making it difficult for development teams to ensure that technical debt is repaid. This paper discusses experiences and best practices to address Technical Debt in an Agile context.

  • Can Large Language Models (LLMs) Compete with Human Requirements Reviewers? – Replication of an Inspection Experiment on Requirements Documents

    by Daniel Seifert, Lisa Jöckel, Adam Trendowicz, Marcus Ciolkowski, Thorsten Honroth, Andreas Jedlitschka


  • Formative Evaluation of a Tool for Managing Software Quality

    by Liliana Guzman, Anna Maria Vollmer, Marcus Ciolkowski, Michael Gillmann

    Context/Background: To achieve high software quality, particularly in the context of agile software development, organizations need tools to continuously analyze software quality. Several quality management (QM) tools have been developed in recent years. However, there is a lack of evidence regarding the quality of QM tools, standardized definitions of such quality, and reliable instruments for measuring it. This, in turn, impedes proper selection and improvement of QM tools. Goals: We aimed at operationalizing the quality of a research QM tool, namely the ProDebt prototype, and evaluating its quality. The goal of the ProDebt prototype is to provide practitioners with support for managing software quality and technical debt. Method: We performed interviews, workshops, and a mapping study to operationalize the quality of the ProDebt prototype and to identify reliable instruments to measure it. We designed a mixed-method study aimed at formative evaluation, i.e., at assessing the quality of the ProDebt prototype and providing guidance for its further development. Eleven practitioners from two German companies evaluated the ProDebt prototype. Results: The participants assessed the information provided by the ProDebt prototype as understandable and relevant. They considered the ProDebt prototype's functionalities as easy to use but of limited usability. They identified improvement needs, e.g., that the analysis results should be linked to other information sources. Conclusions: The evaluation design was of practical value for evaluating the ProDebt prototype considering the limited resources such as the practitioners' time. The evaluation results provided the developers of the ProDebt prototype with guidance for its further development. We conclude that it can be used and tailored for replication or evaluation of other QM tools.


  • Lessons Learned from the ProDebt Research Project on Planning Technical Debt Strategically

    by Marcus Ciolkowski, Liliana Guzmán, Adam Trendowicz, Felix Salfner


  • Open Access

    3-D visualization of dynamic runtime structures

    by Marcus Ciolkowski, Simon Faber, Sebastian von Mammen


  • A Preliminary Survey on Subjective Measurements and Personal Insights into Factors of Perceived Future Project Success

    by Sabine Nunnenmacher, Jessica Jung, Golriz Chehrazi, Alexander Klaus, Constanza Lampasona, Christian Webel, Marcus Ciolkowski

    Within the framework of a project, a variety of known and unknown problems and difficulties in collaboration may occur, which affect how the success of a project is perceived. The goal of our research is to identify factors that influence the perceived success of a project in order to be able to take early countermeasures if project success is threatened. In this paper, we present preliminary results of a study for perceived project success. The study measures perceived project success quantitatively and qualitatively; in addition, our goal was to analyze correlations between perceived project success and aspects of collaboration, communication and individual experiences by using path analysis. The study was conducted in the context of a large-scale research and development project, where multiple partners from research and industry with a heterogeneous background work together to develop new methods and technologies in the field of "internet of things". After the first half of the project, we conducted an explorative survey with 45 persons to evaluate how the current status and project success are perceived by the partners after 1.5 years of collaborative work. As key result, we identified factors that significantly influence the perceived future project success.


  • The QualOSS open source assessment model measuring the performance of open source communities

    by Martin Soto, Marcus Ciolkowski

    Open Source Software (OSS) has an increasing importance for the software industry. Similar to traditional (closed) software acquisition, OSS acquisition requires an assessment of whether its quality is sufficient for the intended purpose, and whether the chances of being maintained and supported in the future, as well as of keeping certain quality standards over time, are sufficiently high. In this paper, we present an approach toward a comprehensive measurement framework for OSS projects, developed in the EU project QualOSS. This approach takes into account product quality as well as process maturity and sustainability of the underlying OSS community.


  • What do we know about perspective-based reading? An approach for quantitative aggregation in software engineering

    by Marcus Ciolkowski

    One of the main challenges in empirical software engineering today lies in the aggregation of evidence. Existing summaries often use qualitative narrative approaches or ad-hoc quantitative methods, such as box plots. With these, information important for decision makers, such as existence and magnitude of a technology's effect, is hard to obtain objectively. Meta-analysis addresses this issue by providing objective quantitative information about a set of studies; however, its usefulness for software engineering studies suffers from high heterogeneity of the studies and missing information. In this paper, we describe an approach for quantitative aggregation of controlled experiments that reduces these two problems. We demonstrate the approach by aggregating available experiments to investigate whether Perspective-Based reading (PBR) improves team effectiveness compared to alternative reading approaches. We then compare the results of our aggregation to previous summaries addressing PBR's team effectiveness. Although the findings are similar, our approach is able to provide the required quantitative information objectively. Our aggregation showed that there is no clear positive effect of PBR: Inspection teams using PBR on requirements documents are more effective when compared to ad-hoc approaches, but are less effective when compared to checklists. In addition, we found strong indicators of researcher bias.