The “Curious Case of Contexts” in Retrieval‐Augmented Generation With a Combination of Labeled and Unlabeled Data
Open Access
- 29 May 2025
- journal article
- Published by Wiley in WIREs Data Mining and Knowledge Discovery
- Vol. 15 (2)
Abstract
With the growing reliance on LLMs for a wide range of NLP tasks, optimizing the use of labeled and unlabeled data for effective context generation has become critical. This work explores the interplay between two prominent methodologies in few‐shot learning: in‐context learning (ICL), which utilizes labeled task‐specific data, and retrieval‐augmented generation (RAG), which leverages unlabeled external knowledge to augment generative models. Since each has its individual limitations, we propose a novel hybrid approach to obtain “the best of both worlds” by dynamically integrating both labeled and unlabeled data towards improving the downstream performance of LLMs. Our methodology, which we call LU‐RAG (labeled and unlabeled RAG), recomputes the scores of top‐k labeled instances and top‐m unlabeled passages to refine context selection. Our experimental results demonstrate that LU‐RAG consistently outperforms both standalone ICL and RAG across multiple benchmarks, showing significant gains in downstream performance. Furthermore, we show that LU‐RAG performs better with a semantic neighborhood as compared to a lexical one, highlighting its ability to generalize effectively.Keywords
This publication has 81 references indexed in Scilit:
- A fine-tuning enhanced RAG system with quantized influence measure as AI judgeScientific Reports, 2024
- “The Absence of Evidence is Not the Evidence of Absence”: Fact Verification via Information Retrieval-Based In-Context Learning2024
- TrumorGPT: Query Optimization and Semantic Reasoning over Networks for Automated Fact-Checking2024
- BashExplainer: Retrieval-Augmented Bash Code Comment Generation based on Fine-tuned CodeBERT2022
- More than a Feeling: Accuracy and Application of Sentiment AnalysisInternational Journal of Research in Marketing, 2022
- A review on the long short-term memory modelArtificial Intelligence Review, 2020
- BERT for Evidence Retrieval and Claim Verification2020
- Comparing automated text classification methodsInternational Journal of Research in Marketing, 2019
- Efficient and Robust Approximate Nearest Neighbor Search Using Hierarchical Navigable Small World GraphsIEEE Transactions on Pattern Analysis and Machine Intelligence, 2018
- Deriving market intelligence from microblogsDecision Support Systems, 2013