TeaRAG: A Token-Efficient Agentic Retrieval-Augmented Generation Framework
Open Access
- 13 June 2026
- journal article
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Information Systems
- Vol. 44 (6) , 1-35
Abstract
Retrieval-Augmented Generation (RAG) utilizes external knowledge to augment Large Language Models’ (LLMs) reliability. For flexibility, agentic RAG employs autonomous, multi-round retrieval and reasoning to resolve queries. Although recent agentic RAG has improved via reinforcement learning, they often incur substantial token overhead from search and reasoning. This trade-off prioritizes accuracy over efficiency. To address this issue, this work proposes TeaRAG, a T oken- e fficient a gentic RAG framework capable of compressing both retrieval content and reasoning steps. 1) First, the retrieved content is compressed by augmenting chunk-based semantic retrieval with a graph retrieval using concise triplets. A knowledge association graph is then built from semantic similarity and co-occurrence. Finally, Personalized PageRank is leveraged to highlight key knowledge within this graph, reducing the number of tokens per retrieval. 2) Besides, to reduce reasoning steps, Iterative Process-aware Direct Preference Optimization (IP-DPO) is proposed. Specifically, our reward function evaluates the knowledge sufficiency by a knowledge matching mechanism, while penalizing excessive reasoning steps. This design can produce high-quality preference-pair datasets, supporting iterative DPO to improve reasoning conciseness. Across six datasets, improves the average Exact Match by \(4\%\) and \(2\%\) while reducing output tokens by \(61\%\) and \(59\%\) on Llama3‑8B‑Instruct and Qwen2.5‑14B‑Instruct, respectively. Code is available at https://github.com/Applied-Machine-Learning-Lab/TeaRAG .
Keywords
Funding Information
- National Science and Technology Major Project (2023ZD0121104)
- Anhui Natural Science Foundation (2508085ZD006)
- National Natural Science Foundation of China (62502404)
- Hong Kong Research Grants Council (R1015-23, C1043-24GF, 11218325)
- Institute of Digital Medicine of City University of Hong Kong (9229503)
- Huawei
- Tencent
- Kuaishou (2025008)
- DIDI
- Bytedance
This publication has 23 references indexed in Scilit:
- Look as You Think: Unifying Reasoning and Visual Evidence Attribution for Verifiable Document RAG via Reinforcement LearningProceedings of the AAAI Conference on Artificial Intelligence, 2026
- Agent4Ranking: Semantic Robust Ranking via Personalized Query Rewriting Using Multi-Agent LLMsACM Transactions on Information Systems, 2025
- WebGLM: Towards an Efficient and Reliable Web-Enhanced Question-Answering SystemACM Transactions on Information Systems, 2025
- Interleaving Retrieval with Chain-of-Thought Reasoning for Knowledge-Intensive Multi-Step Questions2023
- When Not to Trust Language Models: Investigating Effectiveness of Parametric and Non-Parametric Memories2023
- Relation-enhanced Negative Sampling for Multimodal Knowledge Graph Completion2022
- ♫ MuSiQue: Multihop Questions via Single-hop Question CompositionTransactions of the Association for Computational Linguistics, 2022
- Constructing A Multi-hop QA Dataset for Comprehensive Evaluation of Reasoning Steps2020
- Dense Passage Retrieval for Open-Domain Question Answering2020
- Natural Questions: A Benchmark for Question Answering ResearchTransactions of the Association for Computational Linguistics, 2019