LACONIC: Dense-Level Effectiveness for Scalable Sparse Retrieval via a Two-Phase Training Curriculum
Preprint
Abstract
While dense retrieval models have become the standard for state-of-the-art information retrieval, their deployment is often constrained by high memory requirements and reliance on GPU accelerators for vector similarity search. Learned sparse retrieval offers a compelling alternative by enabling efficient search via inverted indices, yet it has historically received less attention than dense approaches. In this report, we introduce LACONIC, a family of learned sparse retrievers based on the Llama-3 architecture (1B, 3B, and 8B). We propose a streamlined two-phase training curriculum consisting of (1) weakly supervised pre-finetuning to adapt causal LLMs for bidirectional contextualization and (2) high-signal finetuning using curated hard negatives. Our results demonstrate that LACONIC effectively bridges the performance gap with dense models: the 8B variant achieves a state-of-the-art 60.2 nDCG on the MTEB Retrieval benchmark, ranking 15th on the leaderboard as of January 1, 2026, while utilizing 71\% less index memory than an equivalent dense model. By delivering high retrieval effectiveness on commodity CPU hardware with a fraction of the compute budget required by competing models, LACONIC provides a scalable and efficient solution for real-world search applications.Keywords
All Related Versions
This publication has 19 references indexed in Scilit:
- Large Language Models for Information Retrieval: A SurveyACM Transactions on Information Systems, 2025
- PyTorch FSDP: Experiences on Scaling Fully Sharded Data ParallelProceedings of the VLDB Endowment, 2023
- Tevatron: An Efficient and Flexible Toolkit for Neural Retrieval2023
- From Distillation to Hard Negative Sampling2022
- An Efficiency Study for SPLADE Models2022
- SPLADE: Sparse Lexical and Expansion Model for First Stage Ranking2021
- Scaling Deep Contrastive Learning Batch Size under Memory Limited Setup2021
- SPARTA: Efficient Open-Domain Question Answering via Sparse Transformer Matching Retrieval2021
- Dense Passage Retrieval for Open-Domain Question Answering2020
- Billion-Scale Similarity Search with GPUsIEEE Transactions on Big Data, 2019