Tilman Kerl
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Tilman Kerl has a total of 9 co-authors. Here are the top co-authors by number of shared publications:
- Elena Leuschner
University of Munster
1 shared publication - Gábor Recski
TU Wien, Vienna, Austria
1 shared publication - Ian Burton
Zeppelin University, Friedrichshafen, Germany
1 shared publication - Ilya Lasy
Center for Artificial Intelligence and Machine Learning TU Wien Vienna Austria
1 shared publication - Mohammad Mahdi Azarbeik
Faculty of Informatics TU Wien
1 shared publication - Rita Sevastjanova
ETH Zurich, Zurich, Switzerland
1 shared publication
Latest Publications
- Open Access
- 1 January 2024
- Published by Association for Computational Linguistics (ACL)
- p. 1183-1196
TU Wien at SemEval-2024 Task 6: Unifying Model-Agnostic and Model-Aware Techniques for Hallucination Detection
Varvara Arzt, Mohammad Mahdi Azarbeik, Ilya Lasy, Tilman Kerl, Gábor Recski. Proceedings of the 18th International Workshop on Semantic Evaluation (SemEval-2024). 2024.
- 1 January 2024
- Published by Association for Computational Linguistics (ACL)
- p. 1183-1196
Open AccessTU Wien at SemEval-2024 Task 6: Unifying Model-Agnostic and Model-Aware Techniques for Hallucination Detection
Varvara Arzt, Mohammad Mahdi Azarbeik, Ilya Lasy, Tilman Kerl, Gábor Recski. Proceedings of the 18th International Workshop on Semantic Evaluation (SemEval-2024). 2024.
- Open Access
- 6 May 2023
- Published by Springer Nature in The Review of International Organizations
- Vol. 18 (4) , 753-776
The performance of international organizations: a new measure and dataset based on computational text analysis of evaluation reports
International organizations (IOs) of the United Nations (UN) system publish around 750 evaluation reports per year, offering insights on their performance across project, program, institutional, and thematic activities. So far, it was not feasible to extract quantitative performance measures from these text-based reports. Using deep learning, this article presents a novel text-based performance metric: We classify individual sentences as containing a negative, positive, or neutral assessment of the evaluated IO activity and then compute the share of positive sentences per report. Content validation yields that the measure adequately reflects the underlying concept of performance; convergent validation finds high correlation with human-provided performance scores by the World Bank; and construct validation shows that our measure has theoretically expected results. Based on this, we present a novel dataset with performance measures for 1,082 evaluated activities implemented by nine UN system IOs and discuss avenues for further research.
- 6 May 2023
- Published by Springer Nature in The Review of International Organizations
- Vol. 18 (4) , 753-776
Open AccessThe performance of international organizations: a new measure and dataset based on computational text analysis of evaluation reports
International organizations (IOs) of the United Nations (UN) system publish around 750 evaluation reports per year, offering insights on their performance across project, program, institutional, and thematic activities. So far, it was not feasible to extract quantitative performance measures from these text-based reports. Using deep learning, this article presents a novel text-based performance metric: We classify individual sentences as containing a negative, positive, or neutral assessment of the evaluated IO activity and then compute the share of positive sentences per report. Content validation yields that the measure adequately reflects the underlying concept of performance; convergent validation finds high correlation with human-provided performance scores by the World Bank; and construct validation shows that our measure has theoretically expected results. Based on this, we present a novel dataset with performance measures for 1,082 evaluated activities implemented by nine UN system IOs and discuss avenues for further research.