Symbolic Chain-of-Thought Distillation: Small Models Can Also "Think" Step-by-Step
Authors:
- Liunian Harold LiLLLiunian Harold Li
- Youngjae YuYYYoungjae Yu
- Xiang RenXRXiang Ren
Preprint
Abstract
Chain-of-thought prompting (e.g., "Let's think step-by-step") primes large language models to verbalize rationalization for their predictions. While chain-of-thought can lead to dramatic performance gains, benefits appear to emerge only for sufficiently large models (beyond 50B parameters). We show that orders-of-magnitude smaller models (125M -- 1.3B parameters) can still benefit from chain-of-thought prompting. To achieve this, we introduce Symbolic Chain-of-Thought Distillation (SCoTD), a method to train a smaller student model on rationalizations sampled from a significantly larger teacher model. Experiments across several commonsense benchmarks show that: 1) SCoTD enhances the performance of the student model in both supervised and few-shot settings, and especially for challenge sets; 2) sampling many reasoning chains per instance from the teacher is paramount; and 3) after distillation, student chain-of-thoughts are judged by humans as comparable to the teacher, despite orders of magnitude fewer parameters. We test several hypotheses regarding what properties of chain-of-thought samples are important, e.g., diversity vs. teacher likelihood vs. open-endedness. We release our corpus of chain-of-thought samples and code.Keywords
All Related Versions
This publication has 21 references indexed in Scilit:
- Large Language Models Can Self-Improve2023
- Large Language Models Are Reasoning Teachers2023
- WANLI: Worker and AI Collaboration for Natural Language Inference Dataset Creation2022
- Reframing Human-AI Collaboration for Generating Free-Text Explanations2022
- MetaICL: Learning to Learn In Context2022
- Symbolic Knowledge Distillation: from General Language Models to Commonsense Models2022
- When Can Models Learn From Explanations? A Formal Framework for Understanding the Roles of Explanation Data2022
- Generating Datasets with Pretrained Language Models2021
- Evaluating Models’ Local Decision Boundaries via Contrast Sets2020
- QUAREL: A Dataset and Models for Answering Questions about Qualitative RelationshipsProceedings of the AAAI Conference on Artificial Intelligence, 2019