MoEfication: Transformer Feed-forward Layers are Mixtures of Experts

Authors:
  • ZZZhengyan Zhang
    Zhengyan Zhang
  • YLYankai Lin
    Yankai Lin
  • ZLZhiyuan Liu
    Zhiyuan Liu
  • PLPeng Li
    Peng Li
  • MSMaosong Sun
    Maosong Sun
  • JZJie Zhou
    Jie Zhou

Top Cited Papers
Open Access
Abstract
Zhengyan Zhang, Yankai Lin, Zhiyuan Liu, Peng Li, Maosong Sun, Jie Zhou. Findings of the Association for Computational Linguistics: ACL 2022. 2022.