Baby Llama: knowledge distillation from an ensemble of teachers trained on a small dataset with no performance penalty
Top Cited Papers
Open Access
- 1 January 2023
- conference paper
- Published by Association for Computational Linguistics (ACL)
- p. 251-261
Abstract
Inar Timiryasov, Jean-Loup Tastet. Proceedings of the BabyLM Challenge at the 27th Conference on Computational Natural Language Learning. 2023.Keywords
All Related Versions
This publication has 8 references indexed in Scilit:
- Predicting Age of Acquisition for Children's Early Vocabulary in Five Languages Using Language Model SurprisalCognitive Science, 2023
- Gradient Ascent Post-training Enhances Language Model Generalization2023
- Do Data-based Curricula Work?2022
- A framework for few-shot language model evaluation2021
- BLiMP: The Benchmark of Linguistic Minimal Pairs for EnglishTransactions of the Association for Computational Linguistics, 2020
- Learning Which Features Matter: RoBERTa Acquires a Preference for Linguistic Generalizations (Eventually)2020
- Mapping the Early Language Environment Using All-Day Recordings and Automated AnalysisAmerican Journal of Speech-Language Pathology, 2017
- Curriculum learning2009