A Study on the Efficiency and Generalization of Light Hybrid Retrievers
Authors:
Preprint
Abstract
Hybrid retrievers can take advantage of both sparse and dense retrievers. Previous hybrid retrievers leverage indexing-heavy dense retrievers. In this work, we study "Is it possible to reduce the indexing memory of hybrid retrievers without sacrificing performance"? Driven by this question, we leverage an indexing-efficient dense retriever (i.e. DrBoost) and introduce a LITE retriever that further reduces the memory of DrBoost. LITE is jointly trained on contrastive learning and knowledge distillation from DrBoost. Then, we integrate BM25, a sparse retriever, with either LITE or DrBoost to form light hybrid retrievers. Our Hybrid-LITE retriever saves 13X memory while maintaining 98.0% performance of the hybrid retriever of BM25 and DPR. In addition, we study the generalization capacity of our light hybrid retrievers on out-of-domain dataset and a set of adversarial attacks datasets. Experiments showcase that light hybrid retrievers achieve better generalization performance than individual sparse and dense retrievers. Nevertheless, our analysis shows that there is a large room to improve the robustness of retrievers, suggesting a new research direction.Keywords
All Related Versions
This publication has 13 references indexed in Scilit:
- CharacterBERT and Self-Teaching for Improving the Robustness of Dense Retrievers on Queries with Typos2022
- Improving Biomedical Information Retrieval with Neural RetrieversProceedings of the AAAI Conference on Artificial Intelligence, 2022
- Salient Phrase Aware Dense Retrieval: Can a Dense Retriever Imitate a Sparse One?2022
- GPL: Generative Pseudo Labeling for Unsupervised Domain Adaptation of Dense Retrieval2022
- Sparse, Dense, and Attentional Representations for Text RetrievalTransactions of the Association for Computational Linguistics, 2021
- Zero-shot Neural Passage Retrieval via Domain-targeted Synthetic Question Generation2021
- TextAttack: A Framework for Adversarial Attacks, Data Augmentation, and Adversarial Training in NLP2020
- Natural Questions: A Benchmark for Question Answering ResearchTransactions of the Association for Computational Linguistics, 2019
- Product Quantization for Nearest Neighbor SearchIEEE Transactions on Pattern Analysis and Machine Intelligence, 2010
- The Probabilistic Relevance Framework: BM25 and BeyondFoundations and Trends® in Information Retrieval, 2009