TRACE: Topical Reasoning with Adaptive Contextual Experts
Open Access
- 13 January 2026
- journal article
- research article
- Published by MDPI AG in Big Data and Cognitive Computing
- Vol. 10 (1) , 31
Abstract
Retrieval-Augmented Generation (RAG) is widely used for long-text summarization due to its efficiency and scalability. However, standard RAG methods flatten documents into independent chunks, disrupting sequential flow and thematic structure, resulting in significant loss of contextual information. This paper presents MOEGAT, a novel graph-enhanced retrieval framework that addresses this limitation by explicitly modeling document structure. MOEGAT constructs an Orthogonal Context Graph to capture sequential discourse and global semantic relationships—long-range dependencies between non-adjacent text spans that reflect topical similarity and logical associations beyond local context. It then employs a query-aware Mixture-of-Experts Graph Attention Network to dynamically activate specialized reasoning pathways. Experiments conducted on three public long-text summarization datasets demonstrate that MOEGAT achieves state-of-the-art performance. Notably, on the WCEP dataset, it outperforms the previous state-of-the-art Graph of Records (GOR) baseline by 14.9%, 18.1%, and 18.4% on ROUGE-L, ROUGE-1, and ROUGE-2, respectively. These substantial gains, especially the 14.9% improvement in ROUGE-L, reflect significantly better capture of long-range coherence and thematic integrity in summaries. Ablation studies confirm the effectiveness of the orthogonal graph and Mixture-of-Experts components. Overall, this work introduces a novel structure-aware approach to RAG that explicitly models and leverages document structure through an orthogonal graph representation and query-aware Mixture-of-Experts reasoning.Keywords
Funding Information
- Zhejiang Key R&D Plan (2017C03047)
This publication has 48 references indexed in Scilit:
- MedSumGraph: enhancing GraphRAG for medical QA with summarization and optimized promptsArtificial Intelligence in Medicine, 2025
- A Knowledge Graph-based RAG for Cross-Document Information Extraction2025
- Knowledge graph enhanced retrieval-augmented generation for failure mode and effects analysisJournal of Industrial Information Integration, 2025
- Graph Retrieval-Augmented Generation for Large Language Models: A Survey2024
- Factual consistency evaluation of summarization in the Era of large language modelsExpert Systems with Applications, 2024
- Almanac — Retrieval-Augmented Language Models for Clinical MedicineNEJM AI, 2024
- AdaMV-MoE: Adaptive Multi-Task Vision Mixture-of-Experts2023
- KEST: A graph-based keyphrase extraction technique for tweets summarization using Markov Decision ProcessExpert Systems with Applications, 2022
- A semantic model of morphological information retrieval: A comparative accumulative analysis2020
- Automatic related work section generation: experiments in scientific document abstractingScientometrics, 2020