The RefinedWeb Dataset for Falcon LLM: Outperforming Curated Corpora with Web Data Only

Authors:
  • GPGuilherme Penedo
    Guilherme Penedo
  • QMQuentin Malartic
    Quentin Malartic
  • DHDaniel Hesslow
    Daniel Hesslow
  • RCRuxandra Cojocaru
    Ruxandra Cojocaru
  • HAHamza Alobeidli
    Hamza Alobeidli
  • ACAlessandro Cappelli
    Alessandro Cappelli
  • BPBaptiste Pannier
    Baptiste Pannier
  • EAEbtesam Almazrouei
    Ebtesam Almazrouei
  • JLJulien Launay
    Julien Launay

Abstract
No abstract available