SPLATE: Sparse Late Interaction Retrieval
Open Access
- 10 July 2024
- conference paper
- Published by Association for Computing Machinery (ACM)
- p. 2635-2640
Abstract
The late interaction paradigm introduced with ColBERT stands out in the neural Information Retrieval space, offering a compelling effectiveness-efficiency trade-off across many benchmarks. Efficient late interaction retrieval is based on an optimized multi-step strategy, where an approximate search first identifies a set of candidate documents to re-rank exactly. In this work, we introduce SPLATE, a simple and lightweight adaptation of the ColBERTv2 model which learns an "MLM adapter'', mapping its frozen token embeddings to a sparse vocabulary space with a partially learned SPLADE module. This allows us to perform the candidate generation step in late interaction pipelines with traditional sparse retrieval techniques, making it particularly appealing for running ColBERT in CPU environments. Our SPLATE ColBERTv2 pipeline achieves the same effectiveness as the PLAID ColBERTv2 engine by re-ranking 50 documents that can be retrieved under 10ms.Keywords
All Related Versions
This publication has 32 references indexed in Scilit:
- Learned Token Pruning in Contextualized Late Interaction over BERT (ColBERT)2022
- From Distillation to Hard Negative Sampling2022
- An Efficiency Study for SPLADE Models2022
- On Approximate Nearest Neighbour Selection for Multi-Stage Dense Retrieval2021
- Query Embedding Pruning for Dense Retrieval2021
- Pseudo-Relevance Feedback for Multiple Representation Dense Retrieval2021
- SPLADE: Sparse Lexical and Expansion Model for First Stage Ranking2021
- A White Box Analysis of ColBERT2021
- ColBERT2020
- Query evaluation: Strategies and optimizationsInformation Processing & Management, 1995