D-CPT Law: Domain-specific Continual Pre-Training Scaling Law for Large Language Models
Authors:
- Haoran QueHQHaoran Que
- Jiaheng LiuJLJiaheng Liu
- Ge ZhangGZGe Zhang
- Chenchen ZhangCZChenchen Zhang
- Xingwei QuXQXingwei Qu
- Yinghao MaYMYinghao Ma
- Feiyu DuanFDFeiyu Duan
- Zhiqi BaiZBZhiqi Bai
- Jiakai WangJWJiakai Wang
- Yuanxing ZhangYZYuanxing Zhang
- Xu TanXTXu Tan
- Jie FuJFJie Fu
- 1 January 2024
- conference paper
- Published by Curran Associates, Inc.
- p. 90318-90354