ToolSandbox: A Stateful, Conversational, Interactive Evaluation Benchmark for LLM Tool Use Capabilities
Authors:
- Jiarui LuJLJiarui Lu
- Thomas HolleisTHThomas Holleis
- Yizhe ZhangYZYizhe Zhang
- Bernhard AumayerBABernhard Aumayer
- Feng NanFNFeng Nan
- Haoping BaiHBHaoping Bai
- Shuang MaSMShuang Ma
- Shen MaSMShen Ma
- Mengyu LiMLMengyu Li
- Guoli YinGYGuoli Yin
- Zirui WangZWZirui Wang
- Ruoming PangRPRuoming Pang
Top Cited Papers
Open Access
- 1 January 2025
- conference paper
- Published by Association for Computational Linguistics (ACL)
- p. 1160-1183
Abstract
Jiarui Lu, Thomas Holleis, Yizhe Zhang, Bernhard Aumayer, Feng Nan, Haoping Bai, Shuang Ma, Shen Ma, Mengyu Li, Guoli Yin, Zirui Wang, Ruoming Pang. Findings of the Association for Computational Linguistics: NAACL 2025. 2025.Keywords
All Related Versions
This publication has 10 references indexed in Scilit:
- AgentBoard: An Analytical Evaluation Board of Multi-turn LLM Agents2024
- API-BLEND: A Comprehensive Corpora for Training and Benchmarking API LLMs2024
- VisualWebArena: Evaluating Multimodal Agents on Realistic Visual Web Tasks2024
- StableToolBench: Towards Stable Large-Scale Benchmarking on Tool Learning of Large Language Models2024
- Probing the Multi-turn Planning Capabilities of LLMs via 20 Question Games2024
- Mind2Web: Towards a Generalist Agent for the Web2023
- API-Bank: A Comprehensive Benchmark for Tool-Augmented LLMs2023
- WebShop: Towards Scalable Real-World Web Interaction with Grounded Language Agents2022
- MultiWOZ - A Large-Scale Multi-Domain Wizard-of-Oz Dataset for Task-Oriented Dialogue Modelling2018
- The Second Dialog State Tracking Challenge2014