The Stability-Efficiency Dilemma: Investigating Sequence Length Warmup for Training GPT Models

Authors:
  • CLConglong Li
    Conglong Li
  • MZMinjia Zhang
    Minjia Zhang
  • YHYuxiong He
    Yuxiong He