SPLADE: Sparse Lexical and Expansion Model for First Stage Ranking
Authors:
Top Cited Papers
Open Access
- 11 July 2021
- conference paper
- Published by Association for Computing Machinery (ACM)
- p. 2288-2292
Abstract
In neural Information Retrieval, ongoing research is directed towards improving the first retriever in ranking pipelines. Learning dense embeddings to conduct retrieval using efficient approximate nearest neighbors methods has proven to work well. Meanwhile, there has been a growing interest in learning sparse representations for documents and queries, that could inherit from the desirable properties of bag-of-words models such as the exact matching of terms and the efficiency of inverted indexes. In this work, we present a new first-stage ranker based on explicit sparsity regularization and a log-saturation effect on term weights, leading to highly sparse representations and competitive results with respect to state-of-the-art dense and sparse methods. Our approach is simple, trained end-to-end in a single stage. We also explore the trade-off between effectiveness and efficiency, by controlling the contribution of the sparsity regularization.Keywords
All Related Versions
This publication has 10 references indexed in Scilit:
- Approximate Nearest Neighbor Search and Lightweight Dense Vector Reranking in Multi-Stage Retrieval Architectures2020
- Expansion via Prediction of Importance with Contextualization2020
- ColBERT2020
- Context-Aware Term Weighting For First Stage Passage Retrieval2020
- Context-Aware Document Term Weighting for Ad-Hoc Search2020
- Critically Examining the "Neural Hype"2019
- Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks2019
- From Neural Re-Ranking to Neural Ranking2018
- Numba2015
- A formal study of information retrieval heuristics2004