Rationale-Guided Retrieval Augmented Generation for Medical Question Answering
Authors:
- Sihyeon ParkSPSihyeon Park
- Mujeen SungMSMujeen Sung
- Jaewoo KangJKJaewoo Kang
Preprint
Abstract
Large language models (LLM) hold significant potential for applications in biomedicine, but they struggle with hallucinations and outdated knowledge. While retrieval-augmented generation (RAG) is generally employed to address these issues, it also has its own set of challenges: (1) LLMs are vulnerable to irrelevant or incorrect context, (2) medical queries are often not well-targeted for helpful information, and (3) retrievers are prone to bias toward the specific source corpus they were trained on. In this study, we present RAG$^2$ (RAtionale-Guided RAG), a new framework for enhancing the reliability of RAG in biomedical contexts. RAG$^2$ incorporates three key innovations: a small filtering model trained on perplexity-based labels of rationales, which selectively augments informative snippets of documents while filtering out distractors; LLM-generated rationales as queries to improve the utility of retrieved snippets; a structure designed to retrieve snippets evenly from a comprehensive set of four biomedical corpora, effectively mitigating retriever bias. Our experiments demonstrate that RAG$^2$ improves the state-of-the-art LLMs of varying sizes, with improvements of up to 6.1\%, and it outperforms the previous best medical RAG model by up to 5.6\% across three medical question-answering benchmarks. Our code is available at https://github.com/dmis-lab/RAG2.Keywords
All Related Versions
This publication has 9 references indexed in Scilit:
- Bailicai: A Domain-Optimized Retrieval-Augmented Generation Framework for Medical ApplicationsBig Data Mining and Analytics, 2026
- Small language models learn enhanced reasoning skills from medical textbooksnpj Digital Medicine, 2025
- SeaKR: Self-aware Knowledge Retrieval for Adaptive Retrieval Augmented Generation2025
- Improving Retrieval-Augmented Generation in Medicine with Iterative Follow-up QuestionsBiocomputing 2020, 2024
- Almanac — Retrieval-Augmented Language Models for Clinical MedicineNEJM AI, 2024
- ClashEval: Quantifying the tug-of-war between an LLM’s internal prior and external evidence2024
- MedCPT: Contrastive Pre-trained Transformers with large-scale PubMed search logs for zero-shot biomedical information retrievalBioinformatics, 2023
- Large language models encode clinical knowledgeNature, 2023
- What Disease Does This Patient Have? A Large-Scale Open Domain Question Answering Dataset from Medical ExamsApplied Sciences, 2021