TableRAG: A Novel Approach for Augmenting LLMs with Information from Retrieved Tables
Open Access
- 17 November 2024
- conference paper
- Published by Sociedade Brasileira de Computação
- p. 182-191
Abstract
We present TableRAG, a novel pipeline designed to integrate tabular data into traditional Retrieval-Augmented Generation (RAG) systems. Our approach is composed of three main parts: (i) generating textual representations of tables; (ii) indexing table representations in vector databases for retrieval, and (iii) employing large language models to generate SQL or Python code for data manipulation over a given table. We assessed the effectiveness of TableRAG by comparing retrieval and re-ranking accuracies over the OTT-QA benchmark and by utilizing both open and closed-source LLMs to generate code for answering questions from the WikiTableQuestions benchmark. Our best results show 86.7% HITS@5 for retrieval and 74% accuracy for Q&A, demonstrating the feasibility of integrating tabular data into RAG systems with high accuracy.Keywords
This publication has 8 references indexed in Scilit:
- A Survey on In-context Learning2024
- Rethinking Tabular Data Understanding with Large Language Models2024
- When Not to Trust Language Models: Investigating Effectiveness of Parametric and Non-Parametric Memories2023
- HybridQA: A Dataset of Multi-Hop Question Answering over Tabular and Textual Data2020
- TaBERT: Pretraining for Joint Understanding of Textual and Tabular Data2020
- Unsupervised Cross-lingual Representation Learning at Scale2020
- Spider: A Large-Scale Human-Labeled Dataset for Complex and Cross-Domain Semantic Parsing and Text-to-SQL Task2018
- Compositional Semantic Parsing on Semi-Structured Tables2015