Thibault Formal
Thibault Formal has a total of 11 co-authors. Here are the top co-authors by number of shared publications:
- Stéphane Clinchant
Naver Labs Europe, Meylan, France
4 shared publications - Carlos Lassance
Cohere
3 shared publications - Benjamin Piwowarski
CNRS, ISIR, Sorbonne Université, Paris, France
2 shared publications - Hervé Déjean
Naver Labs Europe, Meylan, France
2 shared publications - Simon Lupart
University of Amsterdam, Amsterdam, Netherlands
2 shared publications - Guglielmo Faggioli
University of Padua, Padua, Italy
1 shared publication
Latest Publications
- 25 March 2026
- Published by Springer Nature
- p. 470-480
XProvence: Zero-Cost Multilingual Context Pruning for Retrieval-Augmented Generation
- 25 March 2026
- Published by Springer Nature
- p. 470-480
XProvence: Zero-Cost Multilingual Context Pruning for Retrieval-Augmented Generation
- Open Access
- 10 July 2024
- Published by Association for Computing Machinery (ACM)
- p. 2635-2640
SPLATE: Sparse Late Interaction Retrieval
The late interaction paradigm introduced with ColBERT stands out in the neural Information Retrieval space, offering a compelling effectiveness-efficiency trade-off across many benchmarks. Efficient late interaction retrieval is based on an optimized multi-step strategy, where an approximate search first identifies a set of candidate documents to re-rank exactly. In this work, we introduce SPLATE, a simple and lightweight adaptation of the ColBERTv2 model which learns an "MLM adapter'', mapping its frozen token embeddings to a sparse vocabulary space with a partially learned SPLADE module. This allows us to perform the candidate generation step in late interaction pipelines with traditional sparse retrieval techniques, making it particularly appealing for running ColBERT in CPU environments. Our SPLATE ColBERTv2 pipeline achieves the same effectiveness as the PLAID ColBERTv2 engine by re-ranking 50 documents that can be retrieved under 10ms.
- 10 July 2024
- Published by Association for Computing Machinery (ACM)
- p. 2635-2640
Open AccessSPLATE: Sparse Late Interaction Retrieval
The late interaction paradigm introduced with ColBERT stands out in the neural Information Retrieval space, offering a compelling effectiveness-efficiency trade-off across many benchmarks. Efficient late interaction retrieval is based on an optimized multi-step strategy, where an approximate search first identifies a set of candidate documents to re-rank exactly. In this work, we introduce SPLATE, a simple and lightweight adaptation of the ColBERTv2 model which learns an "MLM adapter'', mapping its frozen token embeddings to a sparse vocabulary space with a partially learned SPLADE module. This allows us to perform the candidate generation step in late interaction pipelines with traditional sparse retrieval techniques, making it particularly appealing for running ColBERT in CPU environments. Our SPLATE ColBERTv2 pipeline achieves the same effectiveness as the PLAID ColBERTv2 engine by re-ranking 50 documents that can be retrieved under 10ms.
- Open Access
- 29 April 2024
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Information Systems
- Vol. 42 (5) , 1-46
Towards Effective and Efficient Sparse Neural Information Retrieval
Sparse representation learning based on Pre-trained Language Models has seen a growing interest in Information Retrieval. Such approaches can take advantage of the proven efficiency of inverted indexes and inherit desirable IR priors such as explicit lexical matching or some degree of interpretability. In this work, we thoroughly develop the framework of sparse representation learning in IR, which unifies term weighting and expansion in a supervised setting. We then build on SPLADE—a sparse expansion-based retriever—and show to which extent it is able to benefit from the same training improvements as dense bi-encoders by studying the effect of distillation, hard negative mining, as well as the Pre-trained Language Model’s initialization on itseffectiveness, leading to state-of-the-art results in both in- and out-of-domain evaluation settings (SPLADE++). We furthermore proposeefficiencyimprovements, allowing us to reach latency requirements on par with traditional keyword-based approaches (Efficient-SPLADE).
- 29 April 2024
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Information Systems
- Vol. 42 (5) , 1-46
Open AccessTowards Effective and Efficient Sparse Neural Information Retrieval
Sparse representation learning based on Pre-trained Language Models has seen a growing interest in Information Retrieval. Such approaches can take advantage of the proven efficiency of inverted indexes and inherit desirable IR priors such as explicit lexical matching or some degree of interpretability. In this work, we thoroughly develop the framework of sparse representation learning in IR, which unifies term weighting and expansion in a supervised setting. We then build on SPLADE—a sparse expansion-based retriever—and show to which extent it is able to benefit from the same training improvements as dense bi-encoders by studying the effect of distillation, hard negative mining, as well as the Pre-trained Language Model’s initialization on itseffectiveness, leading to state-of-the-art results in both in- and out-of-domain evaluation settings (SPLADE++). We furthermore proposeefficiencyimprovements, allowing us to reach latency requirements on par with traditional keyword-based approaches (Efficient-SPLADE).
- Open Access
- 9 August 2023
- Published by Association for Computing Machinery (ACM)
- p. 51-63
Towards Query Performance Prediction for Neural Information Retrieval: Challenges and Opportunities
In this work, we propose a novel framework to devise features that can be used by Query Performance Prediction (QPP) models for Neural Information Retrieval (NIR). Using the proposed framework as a periodic table of QPP components, practitioners can devise new predictors better suited for NIR. Through the framework, we detail what challenges and opportunities arise for QPPs at different stages of the NIR pipeline. We show the potential of the proposed framework by using it to devise two types of novel predictors. The first one, named MEMory-based QPP (MEM-QPP), exploits the similarity between test and train queries to measure how much a NIR system can memorize. The second adapts traditional QPPs into NIR-oriented ones by computing the query-corpus semantic similarity. By exploiting the inherent nature of NIR systems, the proposed predictors overcome, under various setups, the current State of the Art, highlighting -- at the same time -- the versatility of the framework in describing different types of QPPs.
- 9 August 2023
- Published by Association for Computing Machinery (ACM)
- p. 51-63
Open AccessTowards Query Performance Prediction for Neural Information Retrieval: Challenges and Opportunities
In this work, we propose a novel framework to devise features that can be used by Query Performance Prediction (QPP) models for Neural Information Retrieval (NIR). Using the proposed framework as a periodic table of QPP components, practitioners can devise new predictors better suited for NIR. Through the framework, we detail what challenges and opportunities arise for QPPs at different stages of the NIR pipeline. We show the potential of the proposed framework by using it to devise two types of novel predictors. The first one, named MEMory-based QPP (MEM-QPP), exploits the similarity between test and train queries to measure how much a NIR system can memorize. The second adapts traditional QPPs into NIR-oriented ones by computing the query-corpus semantic similarity. By exploiting the inherent nature of NIR systems, the proposed predictors overcome, under various setups, the current State of the Art, highlighting -- at the same time -- the versatility of the framework in describing different types of QPPs.
- Open Access
- 18 July 2023
- Published by Association for Computing Machinery (ACM)
- p. 1848-1852
Benchmarking Middle-Trained Language Models for Neural Search
Middle training methods aim to bridge the gap between the Masked Language Model (MLM) pre-training and the final finetuning for retrieval. Recent models such as CoCondenser, RetroMAE, and LexMAE argue that the MLM task is not sufficient enough to pre-train a transformer network for retrieval and hence propose various tasks to do so. Intrigued by those novel methods, we noticed that all these models used different finetuning protocols, making it hard to assess the benefits of middle training. We propose in this paper a benchmark of CoCondenser, RetroMAE, and LexMAE, under the same finetuning conditions. We compare both dense and sparse approaches under various finetuning protocols and middle training on different collections (MS MARCO, Wikipedia). We use additional middle training baselines, such as a standard MLM finetuning on the retrieval collection, optionally augmented by a CLS predicting the passage term frequency. For the sparse approach, our study reveals that there is almost no statistical difference between those methods: the more effective the finetuning procedure is, the less difference there is between those models. For the dense approach, RetroMAE using MS MARCO as middle-training collection shows excellent results in almost all the settings. Finally, we show that middle training on the retrieval collection, thus adapting the language model to it, is a critical factor. Overall, a better experimental setup should be adopted to evaluate middle training methods.
- 18 July 2023
- Published by Association for Computing Machinery (ACM)
- p. 1848-1852
Open AccessBenchmarking Middle-Trained Language Models for Neural Search
Middle training methods aim to bridge the gap between the Masked Language Model (MLM) pre-training and the final finetuning for retrieval. Recent models such as CoCondenser, RetroMAE, and LexMAE argue that the MLM task is not sufficient enough to pre-train a transformer network for retrieval and hence propose various tasks to do so. Intrigued by those novel methods, we noticed that all these models used different finetuning protocols, making it hard to assess the benefits of middle training. We propose in this paper a benchmark of CoCondenser, RetroMAE, and LexMAE, under the same finetuning conditions. We compare both dense and sparse approaches under various finetuning protocols and middle training on different collections (MS MARCO, Wikipedia). We use additional middle training baselines, such as a standard MLM finetuning on the retrieval collection, optionally augmented by a CLS predicting the passage term frequency. For the sparse approach, our study reveals that there is almost no statistical difference between those methods: the more effective the finetuning procedure is, the less difference there is between those models. For the dense approach, RetroMAE using MS MARCO as middle-training collection shows excellent results in almost all the settings. Finally, we show that middle training on the retrieval collection, thus adapting the language model to it, is a critical factor. Overall, a better experimental setup should be adopted to evaluate middle training methods.