Generative Relevance Feedback with Large Language Models
Open Access
- 18 July 2023
- conference paper
- Published by Association for Computing Machinery (ACM)
- p. 2026-2031
Abstract
Current query expansion models use pseudo-relevance feedback to improve first-pass retrieval effectiveness; however, this fails when the initial results are not relevant. Instead of building a language model from retrieved results, we propose Generative Relevance Feedback (GRF) that builds probabilistic feedback models from long-form text generated from Large Language Models. We study the effective methods for generating text by varying the zero-shot generation subtasks: queries, entities, facts, news articles, documents, and essays. We evaluate GRF on document retrieval benchmarks covering a diverse set of queries and document collections, and the results show that GRF methods significantly outperform previous PRF methods. Specifically, we improve MAP between 5-19% and [email protected] 17-24% compared to RM3 expansion, and achieve state-of-the-art recall across all datasets.Keywords
All Related Versions
Funding Information
- Bloomberg L.P. (2019 Bloomberg Data Science Research Grant)
- Engineering and Physical Sciences Research Council (EP/V025708/1)
This publication has 26 references indexed in Scilit:
- Generating Clarifying Questions for Information Retrieval2020
- Query Expansion with Freebase2015
- Entity query feature expansion using knowledge base links2014
- Conceptual language models for domain-specific retrievalInformation Processing & Management, 2010
- Query dependent pseudo-relevance feedback based on wikipedia2009
- Latent concept expansion using markov random fields2007
- A Markov random field model for term dependencies2005
- UMass at TREC 2004: Novelty and HARD2004
- Model-based feedback in the language modeling approach to information retrieval2001
- Some Simple Effective Approximations to the 2-Poisson Model for Probabilistic Weighted Retrieval1994