The RefinedWeb Dataset for Falcon LLM: Outperforming Curated Corpora with Web Data Only
Authors:
- Guilherme PenedoGPGuilherme Penedo
- Quentin MalarticQMQuentin Malartic
- Daniel HesslowDHDaniel Hesslow
- Ruxandra CojocaruRCRuxandra Cojocaru
- Hamza AlobeidliHAHamza Alobeidli
- Alessandro CappelliACAlessandro Cappelli
- Baptiste PannierBPBaptiste Pannier
- Ebtesam AlmazroueiEAEbtesam Almazrouei
- Julien LaunayJLJulien Launay
- 1 January 2023
- conference paper
- Published by Curran Associates, Inc.
- p. 79155-79172