Dynosaur: A Dynamic Growth Paradigm for Instruction-Tuning Data Curation
Preprint
Abstract
Instruction tuning has emerged to enhance the capabilities of large language models (LLMs) to comprehend instructions and generate appropriate responses. Existing methods either manually annotate or employ LLM (e.g., GPT-series) to generate data for instruction tuning. However, they often overlook associating instructions with existing annotated datasets. In this paper, we propose Dynosaur, a dynamic growth paradigm for the automatic curation of instruction-tuning data. Based on the metadata of existing datasets, we use LLMs to automatically construct instruction-tuning data by identifying relevant data fields and generating appropriate instructions. By leveraging the existing annotated datasets, Dynosaur offers several advantages: 1) it reduces the API cost for generating instructions (e.g., it costs less than $12 USD by calling GPT-3.5-turbo for generating 800K instruction tuning samples; 2) it provides high-quality data for instruction tuning (e.g., it performs better than Alpaca and Flan on Super-NI and Longform with comparable data sizes); and 3) it supports the continuous improvement of models by generating instruction-tuning data when a new annotated dataset becomes available. We further investigate a continual learning scheme for learning with the ever-growing instruction-tuning dataset, and demonstrate that replaying tasks with diverse instruction embeddings not only helps mitigate forgetting issues but generalizes to unseen tasks better. Code and data are available at https://github.com/WadeYin9712/Dynosaur.Keywords
All Related Versions
This publication has 8 references indexed in Scilit:
- Instruction Induction: From Few Examples to Natural Language Task Descriptions2023
- Did You Read the Instructions? Rethinking the Effectiveness of Task Definitions in Instruction Learning2023
- Self-Instruct: Aligning Language Models with Self-Generated Instructions2023
- Unnatural Instructions: Tuning Language Models with (Almost) No Human Labor2023
- PromptSource: An Integrated Development Environment and Repository for Natural Language Prompts2022
- Cross-Task Generalization via Natural Language Crowdsourcing Instructions2022
- Datasets: A Community Library for Natural Language Processing2021
- Continual Lifelong Learning in Natural Language Processing: A Survey2020