OneGen: Efficient One-Pass Unified Generation and Retrieval for LLMs
Authors:
- Jintian ZhangJZJintian Zhang
- Mengshu SunMSMengshu Sun
- Jun ZhouJZJun Zhou
- Ningyu ZhangNZNingyu Zhang
Preprint
Abstract
Despite the recent advancements in Large Language Models (LLMs), which have significantly enhanced the generative capabilities for various NLP tasks, LLMs still face limitations in directly handling retrieval tasks. However, many practical applications demand the seamless integration of both retrieval and generation. This paper introduces a novel and efficient One-pass Generation and retrieval framework (OneGen), designed to improve LLMs' performance on tasks that require both generation and retrieval. The proposed framework bridges the traditionally separate training approaches for generation and retrieval by incorporating retrieval tokens generated autoregressively. This enables a single LLM to handle both tasks simultaneously in a unified forward pass. We conduct experiments on two distinct types of composite tasks, RAG and Entity Linking, to validate the pluggability, effectiveness, and efficiency of OneGen in training and inference. Furthermore, our results show that integrating generation and retrieval within the same context preserves the generative capabilities of LLMs while improving retrieval performance. To the best of our knowledge, OneGen is the first to enable LLMs to conduct vector retrieval during the generation.Keywords
All Related Versions
This publication has 31 references indexed in Scilit:
- 🧜Siren’s Song in the AI Ocean: A Survey on Hallucination in Large Language ModelsComputational Linguistics, 2025
- Fine-Tuning LLaMA for Multi-Stage Text Retrieval2024
- Highly Parallel Autoregressive Entity Linking with Discriminative Correction2021
- Scaling Deep Contrastive Learning Batch Size under Memory Limited Setup2021
- ZeRO: Memory optimizations Toward Training Trillion Parameter Models2020
- REL: An Entity Linker Standing on the Shoulders of Giants2020
- Constructing A Multi-hop QA Dataset for Comprehensive Evaluation of Reasoning Steps2020
- Scalable Zero-shot Entity Linking with Dense Entity Retrieval2020
- Open Knowledge Extraction Challenge2015
- KORE2012