Towards enhanced LLM pretraining: Dynamic checkpoint merging via generation quality
Authors:
- Zecheng WangZWZecheng Wang
- Deyuan LiuDLDeyuan Liu
- Chunshan LiCLChunshan Li
- Dianhui ChuDCDianhui Chu
- Weipeng ChenWCWeipeng Chen
- Bingning WangBWBingning Wang
- Dianbo SuiDSDianbo Sui
- 2 July 2025
- journal article
- Published by Elsevier in Information Fusion
- Vol. 125
Abstract
No abstract availableFunding Information
- Natural Science Foundation of Shandong Province (ZR2023QF154)
- National Natural Science Foundation of China (62306087)
- National Key Research and Development Program of China (2023YFB3307500)
- Major Scientific and Technological Innovation Project of Shandong Province
This publication has 12 references indexed in Scilit:
- Rethinking information fusion: Achieving adaptive information throughput and interaction pattern in graph convolutional networks for collaborative filteringInformation Fusion, 2025
- IFIFusion: A independent feature information fusion model for surface defect detectionInformation Fusion, 2025
- Seeing helps hearing: A multi-modal dataset and a mamba-based dual branch parallel network for auditory attention decodingInformation Fusion, 2025
- Information fusion for multi-scale data: Survey and challengesInformation Fusion, 2023
- A Comprehensive Survey on Multimodal Data Representation and Information Fusion Algorithms2021
- Multi-source information fusion based on rough set theory: A reviewInformation Fusion, 2021
- ZeRO: Memory optimizations Toward Training Trillion Parameter Models2020
- Dominant Channel Fusion Architectures - An Intelligent Late Fusion Approach2020
- A survey on machine learning for data fusionInformation Fusion, 2020
- A Decision-Theoretic Generalization of On-Line Learning and an Application to BoostingJournal of Computer and System Sciences, 1997