Abstract
Introduction: Tabletop RPGs (TRPGs) rely heavily on narrative, but Game Masters face challenges creating coherent and engaging dialogues while managing extensive rulebooks. Objective: This work investigates the adoption of Retrieval-Augmented Generation (RAG) to improve dialogue generation in TRPGs using Large Language Models. Methodology or Steps: Four dialogues were analyzed: two handwritten and two generated by LLMs—with and without RAG. Eleven participants rated them across five criteria—Engagement, Coherence, Cohesion, Creativity, and Surprise. Results: RAG-based generation outperformed standard LLMs across all categories, improving coherence (+0.18), cohesion (+0.46), creativity (+0.55), engagement (+0.91), and surprise (+0.64). Compared to handwritten dialogues, generated ones were rated higher in cohesion (3.91 vs. 3.68) and matched in coherence (3.82), although handwritten dialogues remained superior in engagement (3.82 vs. 3.09) and creativity (3.68 vs. 3.14).

This publication has 20 references indexed in Scilit: