Towards enhanced LLM pretraining: Dynamic checkpoint merging via generation quality

Authors:
  • ZWZecheng Wang
    Zecheng Wang
  • DLDeyuan Liu
    Deyuan Liu
  • CLChunshan Li
    Chunshan Li
  • DCDianhui Chu
    Dianhui Chu
  • WCWeipeng Chen
    Weipeng Chen
  • BWBingning Wang
    Bingning Wang
  • DSDianbo Sui
    Dianbo Sui

Abstract
No abstract available
Funding Information
  • Natural Science Foundation of Shandong Province (ZR2023QF154)
  • National Natural Science Foundation of China (62306087)
  • National Key Research and Development Program of China (2023YFB3307500)
  • Major Scientific and Technological Innovation Project of Shandong Province