A Decentralized Retrieval Augmented Generation System with Source Reliabilities Secured on Blockchain
Authors:
- Yining LuYLYining Lu
- Wenyi TangWTWenyi Tang
- Max JohnsonMJMax Johnson
- Taeho JungTJTaeho Jung
- Meng JiangMJMeng Jiang
Preprint
Abstract
Existing retrieval-augmented generation (RAG) systems typically use a centralized architecture, causing a high cost of data collection, integration, and management, as well as privacy concerns. There is a great need for a decentralized RAG system that enables foundation models to utilize information directly from data owners who maintain full control over their sources. However, decentralization brings a challenge: the numerous independent data sources vary significantly in reliability, which can diminish retrieval accuracy and response quality. To address this, our decentralized RAG system has a novel reliability scoring mechanism that dynamically evaluates each source based on the quality of responses it contributes to generate and prioritizes high-quality sources during retrieval. To ensure transparency and trust, the scoring process is securely managed through blockchain-based smart contracts, creating verifiable and tamper-proof reliability records without relying on a central authority. We evaluate our decentralized system with two Llama models (3B and 8B) in two simulated environments where six data sources have different levels of reliability. Our system achieves a +10.7\% performance improvement over its centralized counterpart in the real world-like unreliable data environments. Notably, it approaches the upper-bound performance of centralized systems under ideally reliable data environments. The decentralized infrastructure enables secure and trustworthy scoring management, achieving approximately 56\% marginal cost savings through batched update operations. Our code and system are open-sourced at github.com/yining610/Reliable-dRAG.Keywords
All Related Versions
This publication has 16 references indexed in Scilit:
- The Faiss LibraryIEEE Transactions on Big Data, 2025
- An Architecture and Protocol for Decentralized Retrieval Augmented Generation2025
- LLM-Net: Democratizing LLMs-as-a-Service through Blockchain-based Expert Networks2025
- DeRAG: Decentralized Multi-Source RAG System with Optimized Pyth Network2024
- De-DSI2024
- Seven Failure Points When Engineering a Retrieval Augmented Generation System2024
- M-RAG: Reinforcing Large Language Model Performance through Retrieval-Augmented Generation with Multiple Partitions2024
- RORA: Robust Free-Text Rationale Evaluation2024
- Dense Passage Retrieval for Open-Domain Question Answering2020
- Natural Questions: A Benchmark for Question Answering ResearchTransactions of the Association for Computational Linguistics, 2019