BRBenjamin Roth

Benjamin Roth

*This is an algorithmically generated author record based on publicly available information.

Current Organizations / Institutes

The organizations/institutes from the scholar's latest article

Previous Organizations / Institutes

The organizations / institutes from the scholar's other articles in the past ten years


Benjamin Roth has a total of 52 co-authors. Here are the top co-authors by number of shared publications:


Latest Publications

  • Open Access

    From Models to Systems: A Survey of Explainability for Tool-Augmented Language Models and AI Agents

    by Benjamin Roth, Nicholas Edwards, Pingjun Hong, Loris Schoenegger, Sebastian Schuster

    Large language models (LLMs) are increasingly being used as part of complex agentic systems that orchestrate the use of external tools, such as retrieval mechanisms or code interpreters. In this survey, we argue that this development necessitates a rethinking of the goals of explainable artificial intelligence (XAI): Rather than focusing on providing users with explanations for monolithic machine learning models, we need system-level explanations that also provide information about which and how tools are used, as well as how external execution traces causally influence system behavior. We provide an overview of the existing methods in explainable AI and discuss the limitations of monolithic XAI methods in agentic contexts. Finally, we highlight open challenges in providing faithful explanations for LLM-based systems.

  • Open Access

    Influential Training Data Retrieval for Explaining Verbalized Confidence of LLMs

    by Yuxi Xia, Loris Schoenegger, Benjamin Roth

    Large language models (LLMs) can increase users' perceived trust by verbalizing confidence in their outputs. However, prior work has shown that LLMs are often overconfident, making their stated confidence unreliable since it does not consistently align with factual accuracy. To better understand the sources of this verbalized confidence, we introduce TracVC (\textbf{Trac}ing \textbf{V}erbalized \textbf{C}onfidence), a method that builds on information retrieval and influence estimation to trace generated confidence expressions back to the training data. We evaluate TracVC on OLMo and Llama models in a question answering setting, proposing a new metric, content groundness, which measures the extent to which an LLM grounds its confidence in content-related training examples (relevant to the question and answer) versus in generic examples of confidence verbalization. Our analysis reveals that OLMo2-13B is frequently influenced by confidence-related data that is lexically unrelated to the query, suggesting that it may mimic superficial linguistic expressions of certainty rather than rely on genuine content grounding. These findings point to a fundamental limitation in current training regimes: LLMs may learn how to sound confident without learning when confidence is justified. Our analysis provides a foundation for improving LLMs' trustworthiness in expressing more reliable confidence.

  • Open Access

    Do LLM Self-Explanations Help Users Predict Model Behavior? Evaluating Counterfactual Simulatability with Pragmatic Perturbations

    by Pingjun Hong, Benjamin Roth

    Large Language Models (LLMs) can produce verbalized self-explanations, yet prior studies suggest that such rationales may not reliably reflect the model's true decision process. We ask whether these explanations nevertheless help users predict model behavior, operationalized as counterfactual simulatability. Using StrategyQA, we evaluate how well humans and LLM judges can predict a model's answers to counterfactual follow-up questions, with and without access to the model's chain-of-thought or post-hoc explanations. We compare LLM-generated counterfactuals with pragmatics-based perturbations as alternative ways to construct test cases for assessing the potential usefulness of explanations. Our results show that self-explanations consistently improve simulation accuracy for both LLM judges and humans, but the degree and stability of gains depend strongly on the perturbation strategy and judge strength. We also conduct a qualitative analysis of free-text justifications written by human users when predicting the model's behavior, which provides evidence that access to explanations helps humans form more accurate predictions on the perturbed questions.

  • One and a Half Years of ChatGPT – An Umbrella Review of Large Language Model (LLM) Perception Across Time and Research Fields

    by Leonardo Bergmann, Robin Beckenbach, Lisa Zach, Benjamin Roth, Ulrich S. Tran

    Large language models (LLMs) like ChatGPT have emerged as transformative tools across various research fields. To illustrate the development of perceived benefits and concerns of LLMs across research fields, this umbrella review synthesized findings from 116 systematic reviews. Additionally, we employed LLM-supported data extraction and reported its promising evaluation. Applications in medicine and education dominate the literature, with accuracy and reliability as prominent challenges across disciplines and generally more benefits mentioned than concerns. Thematic analysis of most benefits within the field of psychology revealed LLMs' potentials in empathy and emotional awareness simulation, help with clinical decision-making, and supporting interventions. There were no clear temporal shifts in the perception of LLMs, except for decreasing benefits in ethics and decreasing concerns in education. To inform further research on interdisciplinary applications of LLMs, we have made our full data set consisting of nearly 3,800 extracted benefits and 3,400 concerns publicly available.


  • Open Access

    The Impact of Graph Structure, Cluster Centroid and Text Review Embeddings on Recommendation Methods

    by Peter Dolog, Sergio David Rico Torres, Yllka Velaj, Ylli Sadikaj, Andreas Stephan, Benjamin Roth, Claudia Plant

    It is generally accepted that collaborative information is important for the performance of recommender systems. It is also generally accepted that if this information is sparser, it impacts recommendation systems negatively. Various approaches have tried to lift this problem by employing side information. However, global patterns that can be provided by clusters of similar items and users or even additional information such as text are often not used together with collaborative information. We study the impact of integrating clustering embeddings, review embeddings, and their combinations with embeddings obtained by a recommender system. We study the performance of this approach across various state-of-the-art recommender system algorithms including graph-based methods. We highlight that graph structures are important with sparser datasets and both, in knowledge graphs with side information as well as in collaborative bipartite graphs. In less sparse datasets, a collaborative bipartite graph is usually sufficient. We also highlight that the improvement of recommendation performance through clustering, particularly evident when combined with review embeddings is most visible on sparser data, while on less sparse data incorporating review embeddings may be sufficient when combined with one of the graph-based methods, or otherwise when combined with clustering in other methods.


  • Open Access

    Principled Personas: Defining and Measuring the Intended Effects of Persona Prompting on Task Performance

    by Pedro Henrique Luz de Araujo, Paul Röttger, Dirk Hovy, Benjamin Roth

    Expert persona prompting -- assigning roles such as expert in math to language models -- is widely used for task improvement. However, prior work shows mixed results on its effectiveness, and does not consider when and why personas should improve performance. We analyze the literature on persona prompting for task improvement and distill three desiderata: 1) performance advantage of expert personas, 2) robustness to irrelevant persona attributes, and 3) fidelity to persona attributes. We then evaluate 9 state-of-the-art LLMs across 27 tasks with respect to these desiderata. We find that expert personas usually lead to positive or non-significant performance changes. Surprisingly, models are highly sensitive to irrelevant persona details, with performance drops of almost 30 percentage points. In terms of fidelity, we find that while higher education, specialization, and domain-relatedness can boost performance, their effects are often inconsistent or negligible across tasks. We propose mitigation strategies to improve robustness -- but find they only work for the largest, most capable models. Our findings underscore the need for more careful persona design and for evaluation schemes that reflect the intended effects of persona usage.

  • Open Access

    Principled Personas: Defining and Measuring the Intended Effects of Persona Prompting on Task Performance

    by Pedro Henrique Luz de Araujo, Paul Röttger, Dirk Hovy, Benjamin Roth

    Pedro Henrique Luz de Araujo, Paul Röttger, Dirk Hovy, Benjamin Roth. Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing. 2025.


  • Open Access

    Specification overfitting in artificial intelligence

    by Benjamin Roth, Pedro Henrique Luz de Araujo, Yuxi Xia, Saskia Kaltenbrunner, Christoph Korab

    Machine learning (ML) and artificial intelligence (AI) approaches are often criticized for their inherent bias and for their lack of control, accountability, and transparency. Consequently, regulatory bodies struggle with containing this technology’s potential negative side effects. High-level requirements such as fairness and robustness need to be formalized into concrete specification metrics, imperfect proxies that capture isolated aspects of the underlying requirements. Given possible trade-offs between different metrics and their vulnerability to over-optimization, integrating specification metrics in system development processes is not trivial. This paper defines specification overfitting, a scenario where systems focus excessively on specified metrics to the detriment of high-level requirements and task performance. We present an extensive literature survey to categorize how researchers propose, measure, and optimize specification metrics in several AI fields (e.g., natural language processing, computer vision, reinforcement learning). Using a keyword-based search on papers from major AI conferences and journals between 2018 and mid-2023, we identify and analyze 74 papers that propose or optimize specification metrics. We find that although most papers implicitly address specification overfitting (e.g., by reporting more than one specification metric), they rarely discuss which role specification metrics should play in system development or explicitly define the scope and assumptions behind metric formulations.


  • Open Access

    The Impact of Cluster Centroid and Text Review Embeddings on Recommendation Methods

    by Peter Dolog, Ylli Sadikaj, Yllka Velaj, Andreas Stephan, Benjamin Roth, Claudia Plant