KALE: Using a K-Sparse Projector for Lexical Expansion
Open Access
- 9 August 2023
- conference paper
- Published by Association for Computing Machinery (ACM)
- p. 13-22
Abstract
Recent research has proposed retrieval approaches based on sparse representations and inverted indexes, with terms produced by neural language models and leveraging the advantages from both neural retrieval and lexical matching. This paper proposes KALE, a new lightweight method of this family that uses a small model with a k-sparse projector to convert dense representations into a sparse set of entries from a latent vocabulary. The KALE vocabulary captures semantic concepts than perform well when used in isolation, and perform better when extending the original lexical vocabulary, this way improving first-stage retrieval accuracy. Experiments with the MSMARCOv1 passage retrieval dataset, the TREC Deep Learning dataset, and BEIR datasets, examined the effectiveness of KALE under varying conditions. Results show that the KALE terms can replace the original lexical vocabulary, with gains in accuracy and efficiency. Combining KALE with the original lexical vocabulary, or with other learned terms, can further improve retrieval accuracy with only a modest increase in computational cost.Keywords
Funding Information
- Faculdade de Ciências e Tecnologia, Universidade Nova de Lisboa (SFRH/BD/150497/2019)
This publication has 25 references indexed in Scilit:
- Doc2Query–: When Less is More2023
- A Unified Framework for Learned Sparse Retrieval2023
- Adaptive Batch Scheduling for Open-Domain Question AnsweringIEEE Access, 2021
- Pyserini: A Python Toolkit for Reproducible Information Retrieval Research with Sparse and Dense Representations2021
- ColBERT2020
- From Neural Re-Ranking to Neural Ranking2018
- Efficient Query Processing for Scalable Web Search2018
- Controlled vocabularies and tags: An analysis of research methodsNASKO, 2011
- The Probabilistic Relevance Framework: BM25 and BeyondFoundations and Trends® in Information Retrieval, 2009
- Controlled vocabularies in bioinformatics: a case study in the gene ontologyDrug Discovery Today: BIOSILICO, 2004