Are We Done with MMLU?
Open Access
- 1 January 2025
- conference paper
- Published by Association for Computational Linguistics (ACL)
- p. 5069-5096
Abstract
Aryo Pradipta Gema, Joshua Ong Jun Leang, Giwon Hong, Alessio Devoto, Alberto Carlo Maria Mancino, Rohit Saxena, Xuanli He, Yu Zhao, Xiaotang Du, Mohammad Reza Ghasemi Madani, Claire Barale, Robert McHardy, Joshua Harris, Jean Kaddour, Emile Van Krieken, Pasquale Minervini. Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers). 2025.Keywords
All Related Versions
This publication has 16 references indexed in Scilit:
- Global MMLU: Understanding and Addressing Cultural and Linguistic Biases in Multilingual Evaluation2025
- Croissant: A Metadata Format for ML-Ready Datasets2024
- Deduplicating Training Data Makes Language Models Better2022
- Generating Data to Mitigate Spurious Correlations in Natural Language Inference Datasets2022
- TruthfulQA: Measuring How Models Mimic Human Falsehoods2022
- Toward Annotator Group Bias in Crowdsourcing2022
- Avoiding the Hypothesis-Only Bias in Natural Language Inference via Ensemble Adversarial Training2020
- Human Uncertainty Makes Classification More Robust2019
- Are We Modeling the Task or the Annotator? An Investigation of Annotator Bias in Natural Language Understanding Datasets2019
- Can a Suit of Armor Conduct Electricity? A New Dataset for Open Book Question Answering2018