Re3: Generating Longer Stories With Recursive Reprompting and Revision
Preprint
Abstract
We consider the problem of automatically generating longer stories of over two thousand words. Compared to prior work on shorter stories, long-range plot coherence and relevance are more central challenges here. We propose the Recursive Reprompting and Revision framework (Re3) to address these challenges by (a) prompting a general-purpose language model to construct a structured overarching plan, and (b) generating story passages by repeatedly injecting contextual information from both the plan and current story state into a language model prompt. We then revise by (c) reranking different continuations for plot coherence and premise relevance, and finally (d) editing the best continuation for factual consistency. Compared to similar-length stories generated directly from the same base model, human evaluators judged substantially more of Re3's stories as having a coherent overarching plot (by 14% absolute increase), and relevant to the given initial premise (by 20%).Keywords
All Related Versions
This publication has 9 references indexed in Scilit:
- GeDi: Generative Discriminator Guided Sequence Generation2021
- Dialogue State Tracking with a Language Model using Schema-Driven Prompting2021
- A Knowledge-Enhanced Pretraining Model for Commonsense Story GenerationTransactions of the Association for Computational Linguistics, 2020
- UNIFIEDQA: Crossing Format Boundaries with a Single QA System2020
- Dense Passage Retrieval for Open-Domain Question Answering2020
- Strategies for Structuring Story Generation2019
- Hierarchical Neural Story Generation2018
- Open information extraction from the webCommunications of the ACM, 2008
- Modeling Local Coherence: An Entity-Based ApproachComputational Linguistics, 2008