MMMU-Pro: A More Robust Multi-discipline Multimodal Understanding Benchmark
Authors:
- Xiang YueXYXiang Yue
- Tianyu ZhengTZTianyu Zheng
- Yuansheng NiYNYuansheng Ni
- Yubo WangYWYubo Wang
- Kai ZhangKZKai Zhang
- Shengbang TongSTShengbang Tong
- Yuxuan SunYSYuxuan Sun
- Botao YuBYBotao Yu
- Ge ZhangGZGe Zhang
- Huan SunHSHuan Sun
- Yu SuYSYu Su
- Wenhu ChenWCWenhu Chen
Top Cited Papers
Open Access
- 1 January 2025
- conference paper
- Published by Association for Computational Linguistics (ACL)
- p. 15134-15186
Abstract
Xiang Yue, Tianyu Zheng, Yuansheng Ni, Yubo Wang, Kai Zhang, Shengbang Tong, Yuxuan Sun, Botao Yu, Ge Zhang, Huan Sun, Yu Su, Wenhu Chen, Graham Neubig. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2025.Keywords
All Related Versions
This publication has 8 references indexed in Scilit:
- Efficient multimodal large language models: a surveyVisual Intelligence, 2025
- Otter: A Multi-Modal Model With In-Context Instruction TuningIEEE Transactions on Pattern Analysis and Machine Intelligence, 2025
- BLINK: Multimodal Large Language Models Can See but Not Perceive2024
- MATHVERSE: Does Your Multi-modal LLM Truly See the Diagrams in Visual Math Problems?2024
- Shortcut Learning of Large Language Models in Natural Language UnderstandingCommunications of the ACM, 2023
- VinVL: Revisiting Visual Representations in Vision-Language Models2021
- Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answering2017
- VQA: Visual Question Answering2015