BookWorm: A Dataset for Character Description and Analysis
Preprint
Abstract
Characters are at the heart of every story, driving the plot and engaging readers. In this study, we explore the understanding of characters in full-length books, which contain complex narratives and numerous interacting characters. We define two tasks: character description, which generates a brief factual profile, and character analysis, which offers an in-depth interpretation, including character development, personality, and social context. We introduce the BookWorm dataset, pairing books from the Gutenberg Project with human-written descriptions and analyses. Using this dataset, we evaluate state-of-the-art long-context models in zero-shot and fine-tuning settings, utilizing both retrieval-based and hierarchical processing for book-length inputs. Our findings show that retrieval-based approaches outperform hierarchical ones in both tasks. Additionally, fine-tuned models using coreference-based retrieval produce the most factual descriptions, as measured by fact- and entailment-based metrics. We hope our dataset, experiments, and analysis will inspire further research in character-based narrative understanding.Keywords
All Related Versions
This publication has 28 references indexed in Scilit:
- Conditional Generation with a Question-Answering BlueprintTransactions of the Association for Computational Linguistics, 2023
- Detecting and Grounding Important Characters in Visual StoriesProceedings of the AAAI Conference on Artificial Intelligence, 2023
- SummaC: Re-Visiting NLI-based Models for Inconsistency Detection in SummarizationTransactions of the Association for Computational Linguistics, 2022
- Towards Question-Answering as an Automatic Metric for Evaluating the Content Quality of a SummaryTransactions of the Association for Computational Linguistics, 2021
- SummEval: Re-evaluating Summarization EvaluationTransactions of the Association for Computational Linguistics, 2021
- A Character-Centric Neural Model for Automated Story GenerationProceedings of the AAAI Conference on Artificial Intelligence, 2020
- Modeling Protagonist Emotions for Emotion-Aware Storytelling2020
- Adversarial NLI: A New Benchmark for Natural Language Understanding2020
- Generating Character Descriptions for Automatic Summarization of FictionProceedings of the AAAI Conference on Artificial Intelligence, 2019
- Unsupervised Learning of Evolving Relationships Between Literary CharactersProceedings of the AAAI Conference on Artificial Intelligence, 2017