Anxiang Zeng
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Research Subjects
Anxiang Zeng has a total of 18 co-authors. Here are the top co-authors by number of shared publications:
Haibo Zhang
2 shared publications- Qing Da
Alibaba Group Hangzhou China
2 shared publications - Wenya Zhu
Alibaba (China) Technology Co., Ltd
2 shared publications - Yinfu Feng
Alibaba Group Hangzhou China
2 shared publications - Yinghua Zhang
Hong Kong University of Science and Technology
2 shared publications Baosong Yang
Alibaba (China) Technology Co., Ltd
1 shared publication
Latest Publications
- Open Access
Mid-Training of Large Language Models: A Survey
Large language models (LLMs) are typically developed through large-scale pre-training followed by task-specific fine-tuning. Recent advances highlight the importance of an intermediate mid-training stage, where models undergo multiple annealing-style phases that refine data quality, adapt optimization schedules, and extend context length. This stage mitigates diminishing returns from noisy tokens, stabilizes convergence, and expands model capability in late training. Its effectiveness can be explained through gradient noise scale, the information bottleneck, and curriculum learning, which together promote generalization and abstraction. Despite widespread use in state-of-the-art systems, there has been no prior survey of mid-training as a unified paradigm. We introduce the first taxonomy of LLM mid-training spanning data distribution, learning-rate scheduling, and long-context extension. We distill practical insights, compile evaluation benchmarks, and report gains to enable structured comparisons across models. We also identify open challenges and propose avenues for future research and practice.
Open AccessMid-Training of Large Language Models: A Survey
Large language models (LLMs) are typically developed through large-scale pre-training followed by task-specific fine-tuning. Recent advances highlight the importance of an intermediate mid-training stage, where models undergo multiple annealing-style phases that refine data quality, adapt optimization schedules, and extend context length. This stage mitigates diminishing returns from noisy tokens, stabilizes convergence, and expands model capability in late training. Its effectiveness can be explained through gradient noise scale, the information bottleneck, and curriculum learning, which together promote generalization and abstraction. Despite widespread use in state-of-the-art systems, there has been no prior survey of mid-training as a unified paradigm. We introduce the first taxonomy of LLM mid-training spanning data distribution, learning-rate scheduling, and long-context extension. We distill practical insights, compile evaluation benchmarks, and report gains to enable structured comparisons across models. We also identify open challenges and propose avenues for future research and practice.
- 11 May 2022
- Published by Springer Nature
- p. 458-471
Cross-Lingual Product Retrieval in E-Commerce Search
- 11 May 2022
- Published by Springer Nature
- p. 458-471
Cross-Lingual Product Retrieval in E-Commerce Search
- 10 May 2022
- Published by Springer Nature
- p. 275-287
DHA: Product Title Generation with Discriminative Hierarchical Attention for E-commerce
- 10 May 2022
- Published by Springer Nature
- p. 275-287
DHA: Product Title Generation with Discriminative Hierarchical Attention for E-commerce