Hervé Déjean
Research Subjects
Hervé Déjean has a total of 28 co-authors. Here are the top co-authors by number of shared publications:
- Stéphane Clinchant
Naver Labs Europe, Meylan, France
7 shared publications - Carlos Lassance
Cohere
5 shared publications Jean-Luc Meunier
Xerox Research Centre Europe
3 shared publications- Nicola Tonellotto
University of Pisa, Pisa, Italy
2 shared publications - Simon Lupart
University of Amsterdam, Amsterdam, Netherlands
2 shared publications - Thibault Formal
Naver Labs Europe, Meylan, France
2 shared publications
Latest Publications
- Open Access
- 6 November 2026
- Published by Association for Computing Machinery (ACM)
- p. 783-794
Inference-Free Multimodal Learned Sparse Retrieval for Production-Scale Visual Document Search
- 6 November 2026
- Published by Association for Computing Machinery (ACM)
- p. 783-794
Open AccessInference-Free Multimodal Learned Sparse Retrieval for Production-Scale Visual Document Search
- Open Access
- 10 March 2025
- Published by Association for Computing Machinery (ACM)
- p. 493-502
Context Embeddings for Efficient Answer Generation in Retrieval-Augmented Generation
Retrieval-Augmented Generation (RAG) allows overcoming the limited knowledge of LLMs by extending the input with external information. As a consequence, the contextual inputs to the model become much longer slowing down decoding time affecting the time a user has to wait for an answer. We address this challenge by presenting COCOM, an effective context compression method, reducing long contexts to only a handful of Context Embeddings, speeding up the generation time by a large margin. Our method allows for different compression rates, trading off decoding time for answer quality. Compared to earlier methods, COCOM allows for handling multiple contexts more effectively, significantly reducing decoding time for long inputs. Our method demonstrates an inference speed-up of up to 5.69 times while achieving higher performance compared to existing efficient context compression methods
- 10 March 2025
- Published by Association for Computing Machinery (ACM)
- p. 493-502
Open AccessContext Embeddings for Efficient Answer Generation in Retrieval-Augmented Generation
Retrieval-Augmented Generation (RAG) allows overcoming the limited knowledge of LLMs by extending the input with external information. As a consequence, the contextual inputs to the model become much longer slowing down decoding time affecting the time a user has to wait for an answer. We address this challenge by presenting COCOM, an effective context compression method, reducing long contexts to only a handful of Context Embeddings, speeding up the generation time by a large margin. Our method allows for different compression rates, trading off decoding time for answer quality. Compared to earlier methods, COCOM allows for handling multiple contexts more effectively, significantly reducing decoding time for long inputs. Our method demonstrates an inference speed-up of up to 5.69 times while achieving higher performance compared to existing efficient context compression methods
- Open Access
- 10 July 2024
- Published by Association for Computing Machinery (ACM)
- p. 2635-2640
SPLATE: Sparse Late Interaction Retrieval
The late interaction paradigm introduced with ColBERT stands out in the neural Information Retrieval space, offering a compelling effectiveness-efficiency trade-off across many benchmarks. Efficient late interaction retrieval is based on an optimized multi-step strategy, where an approximate search first identifies a set of candidate documents to re-rank exactly. In this work, we introduce SPLATE, a simple and lightweight adaptation of the ColBERTv2 model which learns an "MLM adapter'', mapping its frozen token embeddings to a sparse vocabulary space with a partially learned SPLADE module. This allows us to perform the candidate generation step in late interaction pipelines with traditional sparse retrieval techniques, making it particularly appealing for running ColBERT in CPU environments. Our SPLATE ColBERTv2 pipeline achieves the same effectiveness as the PLAID ColBERTv2 engine by re-ranking 50 documents that can be retrieved under 10ms.
- 10 July 2024
- Published by Association for Computing Machinery (ACM)
- p. 2635-2640
Open AccessSPLATE: Sparse Late Interaction Retrieval
The late interaction paradigm introduced with ColBERT stands out in the neural Information Retrieval space, offering a compelling effectiveness-efficiency trade-off across many benchmarks. Efficient late interaction retrieval is based on an optimized multi-step strategy, where an approximate search first identifies a set of candidate documents to re-rank exactly. In this work, we introduce SPLATE, a simple and lightweight adaptation of the ColBERTv2 model which learns an "MLM adapter'', mapping its frozen token embeddings to a sparse vocabulary space with a partially learned SPLADE module. This allows us to perform the candidate generation step in late interaction pipelines with traditional sparse retrieval techniques, making it particularly appealing for running ColBERT in CPU environments. Our SPLATE ColBERTv2 pipeline achieves the same effectiveness as the PLAID ColBERTv2 engine by re-ranking 50 documents that can be retrieved under 10ms.
- 16 March 2024
- Published by Springer Nature
- p. 349-363
Two-Step SPLADE: Simple, Efficient and Effective Approximation of SPLADE
- 16 March 2024
- Published by Springer Nature
- p. 349-363
Two-Step SPLADE: Simple, Efficient and Effective Approximation of SPLADE
- Open Access
- 18 July 2023
- Published by Association for Computing Machinery (ACM)
- p. 1848-1852
Benchmarking Middle-Trained Language Models for Neural Search
Middle training methods aim to bridge the gap between the Masked Language Model (MLM) pre-training and the final finetuning for retrieval. Recent models such as CoCondenser, RetroMAE, and LexMAE argue that the MLM task is not sufficient enough to pre-train a transformer network for retrieval and hence propose various tasks to do so. Intrigued by those novel methods, we noticed that all these models used different finetuning protocols, making it hard to assess the benefits of middle training. We propose in this paper a benchmark of CoCondenser, RetroMAE, and LexMAE, under the same finetuning conditions. We compare both dense and sparse approaches under various finetuning protocols and middle training on different collections (MS MARCO, Wikipedia). We use additional middle training baselines, such as a standard MLM finetuning on the retrieval collection, optionally augmented by a CLS predicting the passage term frequency. For the sparse approach, our study reveals that there is almost no statistical difference between those methods: the more effective the finetuning procedure is, the less difference there is between those models. For the dense approach, RetroMAE using MS MARCO as middle-training collection shows excellent results in almost all the settings. Finally, we show that middle training on the retrieval collection, thus adapting the language model to it, is a critical factor. Overall, a better experimental setup should be adopted to evaluate middle training methods.
- 18 July 2023
- Published by Association for Computing Machinery (ACM)
- p. 1848-1852
Open AccessBenchmarking Middle-Trained Language Models for Neural Search
Middle training methods aim to bridge the gap between the Masked Language Model (MLM) pre-training and the final finetuning for retrieval. Recent models such as CoCondenser, RetroMAE, and LexMAE argue that the MLM task is not sufficient enough to pre-train a transformer network for retrieval and hence propose various tasks to do so. Intrigued by those novel methods, we noticed that all these models used different finetuning protocols, making it hard to assess the benefits of middle training. We propose in this paper a benchmark of CoCondenser, RetroMAE, and LexMAE, under the same finetuning conditions. We compare both dense and sparse approaches under various finetuning protocols and middle training on different collections (MS MARCO, Wikipedia). We use additional middle training baselines, such as a standard MLM finetuning on the retrieval collection, optionally augmented by a CLS predicting the passage term frequency. For the sparse approach, our study reveals that there is almost no statistical difference between those methods: the more effective the finetuning procedure is, the less difference there is between those models. For the dense approach, RetroMAE using MS MARCO as middle-training collection shows excellent results in almost all the settings. Finally, we show that middle training on the retrieval collection, thus adapting the language model to it, is a critical factor. Overall, a better experimental setup should be adopted to evaluate middle training methods.
- Open Access
- 18 July 2023
- Published by Association for Computing Machinery (ACM)
- p. 1771-1775
A Static Pruning Study on Sparse Neural Retrievers
- 18 July 2023
- Published by Association for Computing Machinery (ACM)
- p. 1771-1775
Open AccessA Static Pruning Study on Sparse Neural Retrievers
- 17 March 2023
- Published by Springer Nature
- p. 16-31
Parameter-Efficient Sparse Retrievers and Rerankers Using Adapters
- 17 March 2023
- Published by Springer Nature
- p. 16-31
Parameter-Efficient Sparse Retrievers and Rerankers Using Adapters
- Open Access
- 19 May 2014
- Published by Association for Computing Machinery (ACM)
- p. 197-202
Using ancestral layout models for document digitization
In this article, we show how some concepts found in traditional and old layout practices used to layout text (ruling, grid) can improve document digitization. We will first present these basic layout methods, some used since the Antiquity, and explain how some of their key concepts can be 'translated' and used in today's document digitization. In particular, we will show that the traditional concept of type area is a key notion for modeling document layout. An algorithm to compute type area is detailed. We will then illustrate this work with several practical usages and evaluations, from OCR improvement to high-level logical segmentation.
- 19 May 2014
- Published by Association for Computing Machinery (ACM)
- p. 197-202
Open AccessUsing ancestral layout models for document digitization
In this article, we show how some concepts found in traditional and old layout practices used to layout text (ruling, grid) can improve document digitization. We will first present these basic layout methods, some used since the Antiquity, and explain how some of their key concepts can be 'translated' and used in today's document digitization. In particular, we will show that the traditional concept of type area is a key notion for modeling document layout. An algorithm to compute type area is detailed. We will then illustrate this work with several practical usages and evaluations, from OCR improvement to high-level logical segmentation.
- 20 February 2009
- Published by Springer Nature in International Journal on Document Analysis and Recognition (IJDAR)
- Vol. 12 (1) , 1-20
On tables of contents and how to recognize them
- 20 February 2009
- Published by Springer Nature in International Journal on Document Analysis and Recognition (IJDAR)
- Vol. 12 (1) , 1-20
On tables of contents and how to recognize them
- Open Access
- 28 August 2007
- Published by Association for Computing Machinery (ACM)
- p. 135-143
Logical document conversion
- 28 August 2007
- Published by Association for Computing Machinery (ACM)
- p. 135-143
Open AccessLogical document conversion