Sven Schmeier
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Research Subjects
Sven Schmeier has a total of 17 co-authors. Here are the top co-authors by number of shared publications:
- Günter Neumann
DFKI
2 shared publications - Kathrin Eichler
DFKI GmbH
2 shared publications - Akhil Juneja
German Research Center for Artificial Intelligence (DFKI)
1 shared publication - Aljoscha Burchardt
DFKI
1 shared publication - Cindy Tscherwinka
Euroscript Deutschland
1 shared publication - Daniel Röder
German Research Center for Artificial Intelligence (DFKI)
1 shared publication
Latest Publications
- Open Access
Detecting Pipeline Failures through Fine-Grained Analysis of Web Agents
Web agents powered by large language models (LLMs) can autonomously perform complex, multistep tasks in dynamic web environments. However, current evaluations mostly focus on the overall success while overlooking intermediate errors. This limits insight into failure modes and hinders systematic improvement. This work analyzes existing benchmarks and highlights the lack of fine-grained diagnostic tools. To address this gap, we propose a modular evaluation framework that decomposes agent pipelines into interpretable stages for detailed error analysis. Using the SeeAct framework and the Mind2Web dataset as a case study, we show how this approach reveals actionable weaknesses missed by standard metrics - paving the way for more robust and generalizable web agents.
Open AccessDetecting Pipeline Failures through Fine-Grained Analysis of Web Agents
Web agents powered by large language models (LLMs) can autonomously perform complex, multistep tasks in dynamic web environments. However, current evaluations mostly focus on the overall success while overlooking intermediate errors. This limits insight into failure modes and hinders systematic improvement. This work analyzes existing benchmarks and highlights the lack of fine-grained diagnostic tools. To address this gap, we propose a modular evaluation framework that decomposes agent pipelines into interpretable stages for detailed error analysis. Using the SeeAct framework and the Mind2Web dataset as a case study, we show how this approach reveals actionable weaknesses missed by standard metrics - paving the way for more robust and generalizable web agents.
- 15 November 2014
- Published by Springer Nature in Language Resources and Evaluation
- Vol. 48 (4) , 541-559
Involving language professionals in the evaluation of machine translation
- 15 November 2014
- Published by Springer Nature in Language Resources and Evaluation
- Vol. 48 (4) , 541-559
Involving language professionals in the evaluation of machine translation
- 29 September 2012
- Published by Springer Nature in KI - Künstliche Intelligenz
- Vol. 26 (4) , 419-422
Search and Topic Detection in Customer Requests
- 29 September 2012
- Published by Springer Nature in KI - Künstliche Intelligenz
- Vol. 26 (4) , 419-422
Search and Topic Detection in Customer Requests
- 1 January 2012
- Published by INSTICC
- p. 65-74
Guided Exploratory Search on the Mobile Web
- 1 January 2009
- Published by INSTICC
- p. 180-185
DILIA - A DIGITAL LIBRARY ASSISTANT - A New Approach to Information Discovery through Information Extraction and Visualization
- 1 January 2009
- Published by INSTICC
- p. 180-185
DILIA - A DIGITAL LIBRARY ASSISTANT - A New Approach to Information Discovery through Information Extraction and Visualization