Robustness Challenges in Model Distillation and Pruning for Natural Language Understanding
Open Access
- 1 January 2023
- conference paper
- Published by Association for Computational Linguistics (ACL)
- p. 1766-1778
Abstract
Mengnan Du, Subhabrata Mukherjee, Yu Cheng, Milad Shokouhi, Xia Hu, Ahmed Hassan Awadallah. Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics. 2023.Keywords
All Related Versions
This publication has 11 references indexed in Scilit:
- Shortcut Learning of Large Language Models in Natural Language UnderstandingCommunications of the ACM, 2023
- Sparse Progressive Distillation: Resolving Overfitting under Pretrain-and-Finetune Paradigm2022
- Posterior Differential Regularization with f-divergence for Improving Model Robustness2021
- Towards Interpreting and Mitigating Shortcut Learning Behavior of NLU models2021
- An Empirical Study on Robustness to Spurious Correlations using Pre-trained Language ModelsTransactions of the Association for Computational Linguistics, 2020
- BERTs of a feather do not generalize together: Large variability in generalization across models with similar test set performance2020
- PAWS: Paraphrase Adversaries from Word Scrambling2019
- BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding2019
- FEVER: a Large-scale Dataset for Fact Extraction and VERification2018
- Annotation Artifacts in Natural Language Inference Data2018