Distilling Script Knowledge from Large Language Models for Constrained Language Planning
Preprint
Abstract
In everyday life, humans often plan their actions by following step-by-step instructions in the form of goal-oriented scripts. Previous work has exploited language models (LMs) to plan for abstract goals of stereotypical activities (e.g., "make a cake"), but leaves more specific goals with multi-facet constraints understudied (e.g., "make a cake for diabetics"). In this paper, we define the task of constrained language planning for the first time. We propose an overgenerate-then-filter approach to improve large language models (LLMs) on this task, and use it to distill a novel constrained language planning dataset, CoScript, which consists of 55,000 scripts. Empirical results demonstrate that our method significantly improves the constrained language planning ability of LLMs, especially on constraint faithfulness. Furthermore, CoScript is demonstrated to be quite effective in endowing smaller LMs with constrained language planning ability.Keywords
All Related Versions
This publication has 23 references indexed in Scilit:
- Using Language to Generate State Abstractions for Long-Range Planning in Outdoor Environments2022
- proScript: Partially Ordered Scripts Generation2021
- SimCSE: Simple Contrastive Learning of Sentence Embeddings2021
- Probing Toxic Content in Large Pre-Trained Language Models2021
- UNIFIEDQA: Crossing Format Boundaries with a Single QA System2020
- A Dataset for Tracking Entities in Open Domain Procedural Text2020
- Content Planning for Neural Story Generation with Aristotelian Rescoring2020
- Prospection: Interpretable plans from language by predicting the future2019
- Extracting common sense knowledge from text for robot planning2014
- Probase2012