Revisiting Token Dropping Strategy in Efficient BERT Pretraining

Authors:
  • QZQihuang Zhong
    Qihuang Zhong
  • LDLiang Ding
    Liang Ding
  • JLJuhua Liu
    Juhua Liu
  • XLXuebo Liu
    Xuebo Liu
  • MZMin Zhang
    Min Zhang
  • BDBo Du
    Bo Du
  • DTDacheng Tao
    Dacheng Tao

Abstract
Qihuang Zhong, Liang Ding, Juhua Liu, Xuebo Liu, Min Zhang, Bo Du, Dacheng Tao. Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2023.