An Efficiency Study for SPLADE Models
Authors:
Open Access
- 6 July 2022
- conference paper
- Published by Association for Computing Machinery (ACM)
- p. 2220-2226
Abstract
Latency and efficiency issues are often overlooked when evaluating IR models based on Pretrained Language Models (PLMs) in reason of multiple hardware and software testing scenarios. Nevertheless, efficiency is an important part of such systems and should not be overlooked. In this paper, we focus on improving the efficiency of the SPLADE model since it has achieved state-of-the-art zero-shot performance and competitive results on TREC collections. SPLADE efficiency can be controlled via a regularization factor, but solely controlling this regularization has been shown to not be efficient enough. In order to reduce the latency gap between SPLADE and traditional retrieval systems, we propose several techniques including L1 regularization for queries, a separation of document/query encoders, a FLOPS-regularized middle-training, and the use of faster query encoders. Our benchmark demonstrates that we can drastically improve the efficiency of these models while increasing the performance metrics on in-domain data. To our knowledge, we propose the first neural models that, under the same computing constraints, achieve similar latency (less than 4ms difference) as traditional BM25, while having similar performance (less than 10% [email protected] reduction) as the state-of-the-art single-stage neural rankers on in-domain data.Keywords
All Related Versions
This publication has 17 references indexed in Scilit:
- Finding the Best of Both Worlds2020
- Billion-Scale Similarity Search with GPUsIEEE Transactions on Big Data, 2019
- AnseriniJournal of Data and Information Quality, 2018
- Faster BlockMax WAND with Variable-sized Blocks2017
- Anytime Ranking for Impact-Ordered Indexes2015
- Optimizing top-k document retrieval strategies for block-max indexes2013
- Faster top-k document retrieval using block-max indexes2011
- The Probabilistic Relevance Framework: BM25 and BeyondFoundations and Trends® in Information Retrieval, 2009
- Efficient query evaluation using a two-level retrieval process2003
- Query evaluation: Strategies and optimizationsInformation Processing & Management, 1995