Towards Verifiable Text Generation with Generative Agent
Open Access
- 11 April 2025
- journal article
- Published by Association for the Advancement of Artificial Intelligence (AAAI) in Proceedings of the AAAI Conference on Artificial Intelligence
- Vol. 39 (23) , 24230-24238
Abstract
Text generation with citations makes it easy to verify the factuality of Large Language Models’ (LLMs) generations. Existing one-step generation studies expose distinct shortages in answer refinement and in-context demonstration matching. In light of these challenges, we propose R2-MGA, a Retrieval and Reflection Memory-augmented Generative Agent. Specifically, it first retrieves the memory bank to obtain the best-matched memory snippet, then reflects the retrieved snippet as a reasoning rationale, next combines the snippet and the rationale as the best-matched in-context demonstration. Additionally, it is capable of in-depth answer refinement with two specifically designed modules. We evaluate R2-MGA across five LLMs on the ALCE benchmark. The results reveal R2-MGA’ exceptional capabilities in text generation with citations. In particular, compared to the selected baselines, it delivers up to +58.8% and +154.7% relative performance gains on answer correctness and citation quality, respectively. Extensive analyses strongly support the motivations of R2-MGA.Keywords
This publication has 12 references indexed in Scilit:
- From Matching to Generation: A Survey on Generative Information RetrievalACM Transactions on Information Systems, 2025
- Towards Human-centered Proactive Conversational Agents2024
- A survey on large language model based autonomous agentsFrontiers of Computer Science, 2024
- Learning to Retrieve In-Context Examples for Large Language Models2024
- Towards Verifiable Text Generation with Evolving Memory and Self-Reflection2024
- On the Multi-turn Instruction Following for Conversational Web Agents2024
- AutoAct: Automatic Agent Learning from Scratch for QA via Self-Planning2024
- Training Language Models to Generate Text with Citations via Fine-grained Rewards2024
- Effective Large Language Model Adaptation for Improved Grounding and Citation Generation2024
- LLatrieval: LLM-Verified Retrieval for Verifiable Generation2024