Corpus-Steered Query Expansion with Large Language Models
Preprint
Abstract
Recent studies demonstrate that query expansions generated by large language models (LLMs) can considerably enhance information retrieval systems by generating hypothetical documents that answer the queries as expansions. However, challenges arise from misalignments between the expansions and the retrieval corpus, resulting in issues like hallucinations and outdated information due to the limited intrinsic knowledge of LLMs. Inspired by Pseudo Relevance Feedback (PRF), we introduce Corpus-Steered Query Expansion (CSQE) to promote the incorporation of knowledge embedded within the corpus. CSQE utilizes the relevance assessing capability of LLMs to systematically identify pivotal sentences in the initially-retrieved documents. These corpus-originated texts are subsequently used to expand the query together with LLM-knowledge empowered expansions, improving the relevance prediction between the query and the target documents. Extensive experiments reveal that CSQE exhibits strong performance without necessitating any training, especially with queries for which LLMs lack knowledge.Keywords
All Related Versions
This publication has 15 references indexed in Scilit:
- 🧜Siren’s Song in the AI Ocean: A Survey on Hallucination in Large Language ModelsComputational Linguistics, 2025
- Large Language Models can Accurately Predict Searcher Preferences2024
- Perspectives on Large Language Models for Relevance Judgment2023
- Generative Relevance Feedback with Large Language Models2023
- Pyserini: A Python Toolkit for Reproducible Information Retrieval Research with Sparse and Dense Representations2021
- A review of ontology based query expansionInformation Processing & Management, 2007
- Probabilistic models of information retrieval based on measuring the divergence from randomnessACM Transactions on Information Systems, 2002
- Relevance based language models2001
- Concept based query expansion1993
- ON TERM SELECTION FOR QUERY EXPANSIONJournal of Documentation, 1990