A Plan Reuse Mechanism for LLM-Driven Agent
Authors:
- Guopeng LiGLGuopeng Li
- Ruiqi WuRWRuiqi Wu
- Haisheng TanHTHaisheng Tan
Preprint
Abstract
Integrating large language models (LLMs) into personal assistants, like Xiao Ai and Blue Heart V, effectively enhances their ability to interact with humans, solve complex tasks, and manage IoT devices. Such assistants are also termed LLM-driven agents. Upon receiving user requests, the LLM-driven agent generates plans using an LLM, executes these plans through various tools, and then returns the response to the user. During this process, the latency for generating a plan with an LLM can reach tens of seconds, significantly degrading user experience. Real-world dataset analysis shows that about 30% of the requests received by LLM-driven agents are identical or similar, which allows the reuse of previously generated plans to reduce latency. However, it is difficult to accurately define the similarity between the request texts received by the LLM-driven agent through directly evaluating the original request texts. Moreover, the diverse expressions of natural language and the unstructured format of plan texts make implementing plan reuse challenging. To address these issues, we present and implement a plan reuse mechanism for LLM-driven agents called AgentReuse. AgentReuse leverages the similarities and differences among requests' semantics and uses intent classification to evaluate the similarities between requests and enable the reuse of plans. Experimental results based on a real-world dataset demonstrate that AgentReuse achieves a 93% effective plan reuse rate, an F1 score of 0.9718, and an accuracy of 0.9459 in evaluating request similarities, reducing latency by 93.12% compared with baselines without using the reuse mechanism.Keywords
All Related Versions
This publication has 9 references indexed in Scilit:
- HotPrefix: Hotness-Aware KV Cache Scheduling for Efficient Prefix Sharing in LLM Inference SystemsProceedings of the ACM on Management of Data, 2025
- AppAgent: Multimodal Agents as Smartphone Users2025
- The rise and potential of large language model based agents: a surveyScience China Information Sciences, 2025
- Augmenting large language models with chemistry toolsNature Machine Intelligence, 2024
- A survey on large language model based autonomous agentsFrontiers of Computer Science, 2024
- H2O: Heavy-Hitter Oracle for Efficient Generative Inference of Large Language Models2023
- HuggingGPT: Solving AI Tasks with ChatGPT and its Friends in Hugging Face2023
- A Two-Stage Selective Fusion Framework for Joint Intent Detection and Slot FillingIEEE Transactions on Neural Networks and Learning Systems, 2022
- Dependent Function Embedding for Distributed Serverless Edge ComputingIEEE Transactions on Parallel and Distributed Systems, 2021