GenTool: Enhancing Tool Generalization in Language Models through Zero-to-One and Weak-to-Strong Simulation
Preprint
Abstract
Large Language Models (LLMs) can enhance their capabilities as AI assistants by integrating external tools, allowing them to access a wider range of information. While recent LLMs are typically fine-tuned with tool usage examples during supervised fine-tuning (SFT), questions remain about their ability to develop robust tool-usage skills and can effectively generalize to unseen queries and tools. In this work, we present GenTool, a novel training framework that prepares LLMs for diverse generalization challenges in tool utilization. Our approach addresses two fundamental dimensions critical for real-world applications: Zero-to-One Generalization, enabling the model to address queries initially lacking a suitable tool by adopting and utilizing one when it becomes available, and Weak-to-Strong Generalization, allowing models to leverage enhanced versions of existing tools to solve queries. To achieve this, we develop synthetic training data simulating these two dimensions of tool usage and introduce a two-stage fine-tuning approach: optimizing tool ranking, then refining tool selection. Through extensive experiments across four generalization scenarios, we demonstrate that our method significantly enhances the tool-usage capabilities of LLMs ranging from 1B to 8B parameters, achieving performance that surpasses GPT-4o. Furthermore, our analysis also provides valuable insights into the challenges LLMs encounter in tool generalization.Keywords
All Related Versions
This publication has 9 references indexed in Scilit:
- Tool learning with large language models: a surveyFrontiers of Computer Science, 2025
- Confucius: Iterative Tool Learning from Introspection Feedback by Easy-to-Difficult CurriculumProceedings of the AAAI Conference on Artificial Intelligence, 2024
- Chain-of-Verification Reduces Hallucination in Large Language Models2024
- Planning, Creation, Usage: Benchmarking LLMs for Comprehensive Tool Utilization in Real-World Complex Scenarios2024
- Planning and Editing What You Retrieve for Enhanced Tool Learning2024
- Mind2Web: Towards a Generalist Agent for the Web2023
- Language Models can Solve Computer Tasks2023
- Unnatural Instructions: Tuning Language Models with (Almost) No Human Labor2023
- DeepSpeed2020