Comparing the evaluation of the thematic consistency of a symbolic cognitive model and three conversation simulators
Preprint
- 21 April 2026
- preprint
- Published by Elsevier in SSRN Electronic Journal
Abstract
A narrative shows thematic consistency when all events in the plot contribute either to advancing the conflict or to sustaining the coherence of the tale. The Model of Thematic Consistency (MTC) (Pérez y Pérez under review) was designed to identify actions that do not support this consistency in computer-generated narratives. The MTC was realised in MEXICA, a symbolic cognitive model of creative writing (Pérez y Pérez and Sharples, 2001; Pérez y Pérez, 2007). This paper reports a study comparing thematic consistency as assessed by MEXICA with the output of three conversation simulators: Claude, Copilot and DeepSeek. The comparison notes differences in their handling of climactic events, thematic coherence, and emotional states. MEXICA relies on mechanisms based on a computer representation of emotion and conflict, which allow systematic exploration of narrative processes. The simulators, in contrast, operate probabilistically, drawing on statistical patterns from large text corpora. The findings indicate that generative AI can broaden literary analysis by offering varied interpretive scenarios, yet its outputs may be unstable, opaque, and prone to anthropomorphism. Cognitive models such as MEXICA provide clear internal mechanisms and explanatory transparency. These observations underline the value of symbolic cognitive models for the study of narrative in the era of generative AI.Keywords
This publication has 8 references indexed in Scilit:
- An Introduction to Narrative Generators2023
- Story Machines2022
- Predicting Generated Story Quality with Quantitative MeasuresProceedings of the AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment, 2018
- A Standardised Procedure for Evaluating Creative Systems: Computational Creativity Evaluation Based on What it is to be CreativeCognitive Computation, 2012
- Assessing the Novelty of Computer-Generated Narratives Using Empirical MetricsMinds and Machines, 2010
- Employing emotions to drive plot generation in a computer-based storytellerCognitive Systems Research, 2007
- Some Empirical Criteria for Attributing Creativity to a Computer ProgramMinds and Machines, 2007
- MEXICA: A computer model of a cognitive account of creative writingJournal of Experimental & Theoretical Artificial Intelligence, 2001