Idan Pogrebinsky
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Research Subjects
Idan Pogrebinsky has a total of 4 co-authors. Here are the top co-authors by number of shared publications:
- David Carmel
Technology Innovation Institute (TII), Haifa, Israel
2 shared publications - Oren Kurland
Technion, Haifa, Israel
2 shared publications - Oz Huly
Technion, Haifa, Israel
1 shared publication - Yoelle Maarek
Technology Innovation Institute (TII), Haifa, Israel
1 shared publication
Latest Publications
- Open Access
- 18 July 2025
- Published by Association for Computing Machinery (ACM)
- p. 410-415
Enhancing Retrieval-Augmented Generation for Text Completion Through Query Selection
- 18 July 2025
- Published by Association for Computing Machinery (ACM)
- p. 410-415
Open AccessEnhancing Retrieval-Augmented Generation for Text Completion Through Query Selection
- Open Access
- 10 July 2024
- Published by Association for Computing Machinery (ACM)
- p. 2559-2563
Old IR Methods Meet RAG
Retrieval augmented generation (RAG) is an important approach to provide large language models (LLMs) with context pertaining to the text generation task: given a prompt, passages are retrieved from external corpora to ground the generation with more relevant and/or fresher data. Most previous studies used dense retrieval methods for applying RAG in question answering scenarios. However, recent work showed that traditional information retrieval methods (a.k.a. sparse methods) can do as well as or even better than dense retrieval ones. In particular, it was shown that Okapi BM25 outperforms dense retrieval methods, in terms of perplexity, for the fundamental text completion task in LLMs. We extend this study and show, using two popular LLMs, that a broad set of sparse retrieval methods achieve better results than all the dense retrieval methods we experimented with, for varying lengths of queries induced from the prompt. Furthermore, we found that Okapi BM25 is substantially outperformed by a term-proximity retrieval method (MRF), which is in turn outperformed by a pseudo-feedback-based bag-of-terms approach (relevance model). Additional exploration sheds some light on the effectiveness of lexical retrieval methods for RAG. Our findings call for further study of classical retrieval methods for RAG.
- 10 July 2024
- Published by Association for Computing Machinery (ACM)
- p. 2559-2563
Open AccessOld IR Methods Meet RAG
Retrieval augmented generation (RAG) is an important approach to provide large language models (LLMs) with context pertaining to the text generation task: given a prompt, passages are retrieved from external corpora to ground the generation with more relevant and/or fresher data. Most previous studies used dense retrieval methods for applying RAG in question answering scenarios. However, recent work showed that traditional information retrieval methods (a.k.a. sparse methods) can do as well as or even better than dense retrieval ones. In particular, it was shown that Okapi BM25 outperforms dense retrieval methods, in terms of perplexity, for the fundamental text completion task in LLMs. We extend this study and show, using two popular LLMs, that a broad set of sparse retrieval methods achieve better results than all the dense retrieval methods we experimented with, for varying lengths of queries induced from the prompt. Furthermore, we found that Okapi BM25 is substantially outperformed by a term-proximity retrieval method (MRF), which is in turn outperformed by a pseudo-feedback-based bag-of-terms approach (relevance model). Additional exploration sheds some light on the effectiveness of lexical retrieval methods for RAG. Our findings call for further study of classical retrieval methods for RAG.