Data-Efficient Massive Tool Retrieval: A Reinforcement Learning Approach for Query-Tool Alignment with Language Models
Open Access
- 8 December 2024
- conference paper
- Published by Association for Computing Machinery (ACM)
- p. 226-235
Abstract
Recent advancements in large language models (LLMs) integrated with external tools and APIs have successfully addressed complex tasks by using in-context learning or fine-tuning. Despite this progress, the vast scale of tool retrieval remains challenging due to stringent input length constraints. In response, we propose a pre-retrieval strategy from an extensive repository, effectively framing the problem as the massive tool retrieval (MTR) task. We introduce the MTRB (massive tool retrieval benchmark) to evaluate real-world tool-augmented LLM scenarios with a large number of tools. This benchmark is designed for low-resource scenarios and includes a diverse collection of tools with descriptions refined for consistency and clarity. It consists of three subsets, each containing 90 test samples and 10 training samples. To handle the low-resource MTR task, we raise a new query-tool alignment (QTA) framework leverages LLMs to enhance query-tool alignment by rewriting user queries through ranking functions and the direct preference optimization (DPO) method. This approach consistently outperforms existing state-of-the-art models in top-5 and top-10 retrieval tasks across the MTRB benchmark, with improvements up to 93.28% based on the metric Sufficiency@k, which measures the adequacy of tool retrieval within the first k results. Furthermore, ablation studies validate the efficacy of our framework, highlighting its capacity to optimize performance even with limited annotated samples. Specifically, our framework achieves up to 78.53% performance improvement in Sufficiency@k with just a single annotated sample. Additionally, QTA exhibits strong cross-dataset generalizability, emphasizing its potential for real-world applications.Keywords
All Related Versions
This publication has 11 references indexed in Scilit:
- EASYTOOL: Enhancing LLM-based Agents with Concise Tool Instruction2025
- API-Bank: A Comprehensive Benchmark for Tool-Augmented LLMs2023
- OpenICL: An Open-Source Framework for In-context Learning2023
- A Sentence is Worth 128 Pseudo Tokens: A Semantic-Aware Contrastive Learning Framework for Sentence Embeddings2022
- MiniLMv2: Multi-Head Self-Attention Relation Distillation for Compressing Pretrained Transformers2021
- BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding2019
- Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks2019
- LETOR: A benchmark collection for research on learning to rank for information retrievalInformation Retrieval Journal, 2010
- The Probabilistic Relevance Framework: BM25 and BeyondFoundations and Trends® in Information Retrieval, 2009
- Cumulated gain-based evaluation of IR techniquesACM Transactions on Information Systems, 2002