LiNR: Model Based Neural Retrieval on GPUs at LinkedIn
Authors:
- Fedor BorisyukFBFedor Borisyuk
- Qingquan SongQSQingquan Song
- Mingzhou ZhouMZMingzhou Zhou
- Ganesh ParameswaranGPGanesh Parameswaran
- Madhu ArunMAMadhu Arun
- Siva PopuriSPSiva Popuri
- Tugrul BingolTBTugrul Bingol
- Zhuotao PeiZPZhuotao Pei
- Kuang-Hsuan LeeKLKuang-Hsuan Lee
- Lu ZhengLZLu Zheng
- Qizhan ShaoQSQizhan Shao
- Ali NaqviANAli Naqvi
Open Access
- 21 October 2024
- conference paper
- Published by Association for Computing Machinery (ACM)
- p. 4366-4373
Abstract
This paper introduces LiNR, LinkedIn's large-scale, GPU-based retrieval system. LiNR supports a billion-sized index on GPU models. We discuss our experiences and challenges in creating scalable, differentiable search indexes using TensorFlow and PyTorch at production scale. In LiNR, both items and model weights are integrated into the model binary. Viewing index construction as a form of model training, we describe scaling our system for large indexes, incorporating full scans and efficient filtering. A key focus is on enabling attribute-based pre-filtering for exhaustive GPU searches, addressing the common challenge of post-filtering in KNN searches that often reduces system quality. We further provide multi-embedding retrieval algorithms and strategies for tackling cold start issues in retrieval. Our advancements in supporting larger indexes through quantization are also discussed. We believe LiNR represents one of the industry's first Live-updated model-based retrieval indexes. Applied to out-of-network post recommendations on LinkedIn Feed, LiNR has contributed to a 3% relative increase in professional daily active users. We envisage LiNR as a step towards integrating retrieval and ranking into a single GPU model, simplifying complex infrastructures and enabling end-to-end optimization of the entire differentiable infrastructure through gradient descent.Keywords
All Related Versions
This publication has 10 references indexed in Scilit:
- LiGNN: Graph Neural Networks at LinkedIn2024
- Revisiting Neural Retrieval on Accelerators2023
- NxtPost: User To Post Recommendations In Facebook Groups2022
- Que2Search: Fast and Accurate Query and Document Understanding for Search at Facebook2021
- Embedding-based Retrieval in Facebook Search2020
- XDL2019
- Billion-Scale Similarity Search with GPUsIEEE Transactions on Big Data, 2019
- Efficient and Robust Approximate Nearest Neighbor Search Using Hierarchical Navigable Small World GraphsIEEE Transactions on Pattern Analysis and Machine Intelligence, 2018
- UnicornProceedings of the VLDB Endowment, 2013
- Product Quantization for Nearest Neighbor SearchIEEE Transactions on Pattern Analysis and Machine Intelligence, 2010