DSDominik Stammbach

Dominik Stammbach

*This is an algorithmically generated author record based on publicly available information.

Current Organizations / Institutes

The organizations/institutes from the scholar's latest article


Dominik Stammbach has a total of 49 co-authors. Here are the top co-authors by number of shared publications:


Latest Publications

  • Open Access

    AI-Assisted Moot Courts: Simulating Justice-Specific Questioning in Oral Arguments

    by Kylie Zhang, Nimra Nadeem, Lucia Zheng, Dominik Stammbach, Peter Henderson

    In oral arguments, judges probe attorneys with questions about the factual record, legal claims, and the strength of their arguments. To prepare for this questioning, both law schools and practicing attorneys rely on moot courts: practice simulations of appellate hearings. Leveraging a dataset of U.S. Supreme Court oral argument transcripts, we examine whether AI models can effectively simulate justice-specific questioning for moot court-style training. Evaluating oral argument simulation is challenging because there is no single correct question for any given turn. Instead, effective questioning should reflect a combination of desirable qualities, such as anticipating substantive legal issues, detecting logical weaknesses, and maintaining an appropriately adversarial tone. We introduce a two-layer evaluation framework that assesses both the realism and pedagogical usefulness of simulated questions using complementary proxy metrics. We construct and evaluate both prompt-based and agentic oral argument simulators. We find that simulated questions are often perceived as realistic by human annotators and achieve high recall of ground truth substantive legal issues. However, models still face substantial shortcomings, including low diversity in question types and sycophancy. Importantly, these shortcomings would remain undetected under naive evaluation approaches.

  • Open Access

    Evaluating GPT-4’s ability to generate informed consent material for genetic testing

    by Eirini Petrou, Kelly E. Ormond, Dominik Stammbach, Elliott Ash, Oliver Buchholz, Effy Vayena

    The informed consent (IC) process is essential in genetic testing, yet IC materials are often difficult to read and understand, influencing patients’ decision-making. Large language models may improve the accessibility and clarity of these materials. We used GPT-4 to generate IC materials for Non-Invasive Prenatal Testing (NIPT) and hereditary breast and ovarian cancer testing (BRCA) in English, German, Italian, and Greek, using zero-shot prompting and retrieval-augmented generation. Healthcare providers evaluated GPT-4-generated and human-generated materials using a previously published framework. GPT-4 performed well on structured components, such as explaining the purpose and benefits of testing, but struggled with nuanced ethical and contextual content. Respondents overall preferred human-written materials, underscoring limitations in current GPT-4-generated material for health communication in the genetic testing context. GPT-4’s performance in German, Italian and Greek was generally weaker than in English, highlighting potential language-specific challenges in GPT-4-generated IC content.


  • Top Cited Papers
    Open Access

    Automated fact-checking of climate claims with large language models

    by Markus Leippold, Saeid Ashraf Vaghefi, Dominik Stammbach, Veruska Muccione, Julia Bingler, Jingwei Ni, Chiara Colesanti Senni, Tobias Wekhof, Tobias Schimanski, Glen Gostlow, et al.

    Accurate identification of true versus false climate information in the digital age is critical. Misinformation can significantly affect public understanding and policymaking. Automated fact-checking seeks to validate claims against trustworthy factual data. This study tackles the challenge of fact-checking climate claims by leveraging the currently most capable Large Language Models (LLMs). To this end, we introduce Climinator, an acronym for CLImate Mediator for INformed Analysis and Transparent Objective Reasoning. It significantly boosts the performance of automated fact-checking by integrating authoritative, up-to-date sources within a novel debating framework. This framework provides a trustworthy and context-aware analysis incorporating multiple scientific viewpoints. Climinator helps identify misinformation in real time and facilitates informed dialog on climate change, highlighting AI’s role in environmental discussions and policy with reliable data.


  • Open Access

    Integrating artificial intelligence with expert knowledge in global environmental assessments: opportunities, challenges and the way ahead

    by Veruska Muccione, Saeid Ashraf Vaghefi, Julia Bingler, Simon K. Allen, Mathias Kraus, Glen Gostlow, Tobias Wekhof, Chiara Colesanti-Senni, Dominik Stammbach, Jingwei Ni, et al.

    With new cycles of global environmental assessments (GEAs) recently starting, including GEO-7 and IPCC AR7, there is increasing need for artificial intelligence (AI) to support in synthesising the rapidly growing body of evidence for authors and users of these assessments. In this article, we explore recent advances in AI and connect them to the different stages of GEAs showing how some processes can be automatised and streamlined. The meticulous and labour-intensive nature of GEAs serves as both a valuable strength and a challenge to staying pertinent and current in today’s era of urgency and the pursuit of the latest knowledge. Utilising AI tools for reviewing and synthesizing scientific literature holds the evident promise of substantially lessening the workload for experts and expediting the assessment process. This, in turn, could lead to more frequent report releases and a smoother integration of the latest scientific advancements into actionable measures. However, successful outcomes can only be achieved if domain experts co-develop and oversee the deployment of such tools together with AI researchers. Otherwise, these tools run the risk of producing inaccurate, incomplete, or misleading information with significant consequences. We demonstrate this through a few examples that compare recently deployed large language models (LLMs) based tools in their performance in capturing nuanced concepts in the context of the reports of the Intergovernmental Panel on Climate Change (IPCC). We recommend establishing ethical committees and organising dedicated expert meetings to develop best practice guidelines, ensuring responsible and transparent integration of AI into GEAs.


  • Open Access

    Translating Legalese

    by Elliott Ash, Aniket Kesari, Suresh Naidu, Lena Song, Dominik Stammbach

    Judicial opinions are written to be persuasive and could build public trust in court decisions, yet they can be difficult for non-experts to understand. We present a pipeline for using an AI assistant to generate simplified summaries of judicial opinions. Compared to existing expert-written summaries, these AI-generated simple summaries are more accessible to the public and more easily understood by non-experts. We show in a survey experiment that the AI summaries help respondents understand the key features of a ruling, and have higher perceived quality, especially for respondents with less formal education.


  • Top Cited Papers
    Open Access

    ChatClimate: Grounding conversational AI in climate science

    by Saeid Ashraf Vaghefi, Dominik Stammbach, Veruska Muccione, Julia Bingler, Jingwei Ni, Mathias Kraus, Simon Allen, Chiara Colesanti-Senni, Tobias Wekhof, Tobias Schimanski, et al.

    Large Language Models have made remarkable progress in question-answering tasks, but challenges like hallucination and outdated information persist. These issues are especially critical in domains like climate change, where timely access to reliable information is vital. One solution is granting these models access to external, scientifically accurate sources to enhance their knowledge and reliability. Here, we enhance GPT-4 by providing access to the Sixth Assessment Report of the Intergovernmental Panel on Climate Change (IPCC AR6), the most comprehensive, up-to-date, and reliable source in this domain (refer to the ’Data Availability’ section). We present our conversational AI prototype, available atwww.chatclimate.ai, and demonstrate its ability to answer challenging questions in three different setups: (1) GPT-4, (2) ChatClimate, which relies exclusively on IPCC AR6 reports, and (3) Hybrid ChatClimate, which utilizes IPCC AR6 reports with in-house GPT-4 knowledge. The evaluation of answers by experts show that the hybrid ChatClimate AI assistant provide more accurate responses, highlighting the effectiveness of our solution.


  • Open Access

    ChatClimate: Grounding Conversational AI in Climate Science

    by Veruska Muccione, Dominik Stammbach, Jingwei Ni, Mathias Kraus, Julia Bingler, Simon Allen, Chiara Colesanti-Senni, Tobias Wekhof, Tobias Schimanski, Glen Gostlow, et al.

    Large Language Models (LLMs) have made significant progress in recent years, achieving remarkable results in question-answering tasks (QA). However, they still face two major challenges at inference time: hallucination and outdated information. These challenges take center stage in critical domains like climate change, where obtaining accurate and up-to-date information from reliable sources in a timely fashion is essential. To overcome these barriers, one potential solution is to provide LLMs with access to external, scientifically accurate, and robust sources (long-term memory). This access assists LLMs in continuously updating their knowledge and preventing the propagation of inaccurate, incorrect, or outdated information. In this study, we enhanced GPT-4 by providing it access to the Sixth Assessment Report of the Intergovernmental (IPCC-AR6), the most comprehensive, up-to-date, and reliable source in this domain. We present our conversational AI prototype, available at www.chatclimate.ai. The product answers challenging questions accurately in three different QA scenarios: asking from 1) GPT-4, 2) ChatClimate, and 3) hybrid ChatClimate. The answers and their sources were evaluated by a team of IPCC authors, who used their expert knowledge to score the accuracy of the answers from 1 (very-low) to 5 (very-high). The evaluation showed that the Standalone assistant provided more accurate answers.


  • Open Access

    Enhancing Large Language Models with Climate Resources

    by Mathias Kraus, Julia Anna Bingler, Markus Leippold, Tobias Schimanski, Chiara Colesanti Senni, Dominik Stammbach, Saeid Ashraf Vaghefi, Nicolas Webersinke

    Large language models (LLMs) have significantly transformed the landscape of artificial intelligence by demonstrating their ability in generating human-like text across diverse topics. However, despite their impressive capabilities, LLMs lack recent information and often employ imprecise language, which can be detrimental in domains where accuracy is crucial, such as climate change. In this study, we make use of recent ideas to harness the potential of LLMs by viewing them as agents that access multiple sources, including databases containing recent and precise information about organizations, institutions, and companies. We demonstrate the effectiveness of our method through a prototype agent that retrieves emission data from ClimateWatch (https://www.climatewatchdata.org/) and leverages general Google search. By integrating these resources with LLMs, our approach overcomes the limitations associated with imprecise language and delivers more reliable and accurate information in the critical domain of climate change. This work paves the way for future advancements in LLMs and their application in domains where precision is of paramount importance.