Rephrasing the Web: A Recipe for Compute and Data-Efficient Language Modeling
Authors:
- Pratyush MainiPMPratyush Maini
- Skyler SetoSSSkyler Seto
- Richard BaiRBRichard Bai
- David GrangierDGDavid Grangier
- Yizhe ZhangYZYizhe Zhang
- Navdeep JaitlyNJNavdeep Jaitly
Top Cited Papers
Open Access
- 1 January 2024
- conference paper
- Published by Association for Computational Linguistics (ACL)
- p. 14044-14072
Abstract
Pratyush Maini, Skyler Seto, Richard Bai, David Grangier, Yizhe Zhang, Navdeep Jaitly. Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2024.Keywords
All Related Versions
This publication has 7 references indexed in Scilit:
- Unlocking Efficiency in Large Language Model Inference: A Comprehensive Survey of Speculative Decoding2024
- Self-Consuming Generative Models go MAD2023
- LogiQA 2.0—An Improved Dataset for Logical Reasoning in Natural Language UnderstandingIEEE/ACM Transactions on Audio, Speech, and Language Processing, 2023
- TruthfulQA: Measuring How Models Mimic Human Falsehoods2022
- Can a Suit of Armor Conduct Electricity? A New Dataset for Open Book Question Answering2018
- Crowdsourcing Multiple Choice Science Questions2017
- The Dependency Locality Theory: A Distance-Based Theory of Linguistic Complexity2001