Are Pretrained Convolutions Better than Pretrained Transformers?

Authors:
  • YTYi Tay
    Yi Tay
  • MDMostafa Dehghani
    Mostafa Dehghani
  • JGJai Prakash Gupta
    Jai Prakash Gupta
  • VAVamsi Aribandi
    Vamsi Aribandi
  • DBDara Bahri
    Dara Bahri
  • ZQZhen Qin
    Zhen Qin
  • DMDonald Metzler
    Donald Metzler
Open Access
Publisher website
Google Scholar
Abstract
Yi Tay, Mostafa Dehghani, Jai Prakash Gupta, Vamsi Aribandi, Dara Bahri, Zhen Qin, Donald Metzler. Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers). 2021.