A Pretrainer's Guide to Training Data: Measuring the Effects of Data Age, Domain Coverage, Quality, & Toxicity
Authors:
- Shayne LongpreSLShayne Longpre
- Gregory YauneyGYGregory Yauney
- Emily ReifEREmily Reif
- Katherine LeeKLKatherine Lee
- Adam RobertsARAdam Roberts
- Barret ZophBZBarret Zoph
- Denny ZhouDZDenny Zhou
- Jason WeiJWJason Wei
- Kevin RobinsonKRKevin Robinson
- David MimnoDMDavid Mimno
- Daphne IppolitoDIDaphne Ippolito
Preprint