RankFlow: A Multi-Role Collaborative Reranking Workflow Utilizing Large Language Models
Preprint
Abstract
In an Information Retrieval (IR) system, reranking plays a critical role by sorting candidate passages according to their relevance to a specific query. This process demands a nuanced understanding of the variations among passages linked to the query. In this work, we introduce RankFlow, a multi-role reranking workflow that leverages the capabilities of Large Language Models (LLMs) and role specializations to improve reranking performance. RankFlow enlists LLMs to fulfill four distinct roles: the query Rewriter, the pseudo Answerer, the passage Summarizer, and the Reranker. This orchestrated approach enables RankFlow to: (1) accurately interpret queries, (2) draw upon LLMs' extensive pre-existing knowledge, (3) distill passages into concise versions, and (4) assess passages in a comprehensive manner, resulting in notably better reranking results. Our experimental results reveal that RankFlow outperforms existing leading approaches on widely recognized IR benchmarks, such as TREC-DL, BEIR, and NovelEval. Additionally, we investigate the individual contributions of each role in RankFlow. Code is available at https://github.com/jincan333/RankFlow.Keywords
All Related Versions
This publication has 14 references indexed in Scilit:
- Large Language Models for Information Retrieval: A SurveyACM Transactions on Information Systems, 2025
- APEER : Automatic Prompt Engineering Enhances Large Language Model Reranking2025
- Visual Prompting Upgrades Neural Network Sparsification: A Data-Model PerspectiveProceedings of the AAAI Conference on Artificial Intelligence, 2025
- AdaPI: Facilitating DNN Model Adaptivity for Efficient Private Inference in Edge Computing2024
- Towards Open-World Recommendation with Knowledge Augmentation from Large Language Models2024
- Recommender Systems in the Era of Large Language Models (LLMs)IEEE Transactions on Knowledge and Data Engineering, 2024
- Same Task, More Tokens: the Impact of Input Length on the Reasoning Performance of Large Language Models2024
- Large Language Models are Effective Text Rankers with Pairwise Ranking Prompting2024
- Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language ProcessingACM Computing Surveys, 2023
- UMass at TREC 2004: Novelty and HARD2004