Utilizing Metadata for Better Retrieval-Augmented Generation
Authors:
- Raquib Bin YousufRYRaquib Bin Yousuf
- Mandar SharmaMSMandar Sharma
- Andrew NeeserANAndrew Neeser
- Chris LatimerCLChris Latimer
- Naren RamakrishnanNRNaren Ramakrishnan
Preprint
Abstract
Retrieval-Augmented Generation systems depend on retrieving semantically relevant document chunks to support accurate, grounded outputs from large language models. In structured and repetitive corpora such as regulatory filings, chunk similarity alone often fails to distinguish between documents with overlapping language. Practitioners often flatten metadata into input text as a heuristic, but the impact and trade-offs of this practice remain poorly understood. We present a systematic study of metadata-aware retrieval strategies, comparing plain-text baselines with approaches that embed metadata directly. Our evaluation spans metadata-as-text (prefix and suffix), a dual-encoder unified embedding that fuses metadata and content in a single index, dual-encoder late-fusion retrieval, and metadata-aware query reformulation. Across multiple retrieval metrics and question types, we find that prefixing and unified embeddings consistently outperform plain-text baselines, with the unified at times exceeding prefixing while being easier to maintain. Beyond empirical comparisons, we analyze embedding space, showing that metadata integration improves effectiveness by increasing intra-document cohesion, reducing inter-document confusion, and widening the separation between relevant and irrelevant chunks. Field-level ablations show that structural cues provide strong disambiguating signals. Our code, evaluation framework, and the RAGMATE-10K dataset are publicly hosted.Keywords
All Related Versions
This publication has 4 references indexed in Scilit:
- Almanac — Retrieval-Augmented Language Models for Clinical MedicineNEJM AI, 2024
- FinBen: A Holistic Financial Benchmark for Large Language Models2024
- Is GPT-4 a reliable rater? Evaluating consistency in GPT-4's text ratingsFrontiers in Education, 2023
- Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena2023