Multimodal Large Language Models for Text-rich Image Understanding: A Comprehensive Review
Authors:
- Pei FuPFPei Fu
- Tongkun GuanTGTongkun Guan
- Zining WangZWZining Wang
- Zhentao GuoZGZhentao Guo
- Chen DuanCDChen Duan
- Hao SunHSHao Sun
- Boming ChenBCBoming Chen
- Jiayao MaJMJiayao Ma
- Qianyi JiangQJQianyi Jiang
- Kai ZhouKZKai Zhou
- Junfeng LuoJLJunfeng Luo
Preprint
Abstract
The recent emergence of Multi-modal Large Language Models (MLLMs) has introduced a new dimension to the Text-rich Image Understanding (TIU) field, with models demonstrating impressive and inspiring performance. However, their rapid evolution and widespread adoption have made it increasingly challenging to keep up with the latest advancements. To address this, we present a systematic and comprehensive survey to facilitate further research on TIU MLLMs. Initially, we outline the timeline, architecture, and pipeline of nearly all TIU MLLMs. Then, we review the performance of selected models on mainstream benchmarks. Finally, we explore promising directions, challenges, and limitations within the field.Keywords
All Related Versions
This publication has 42 references indexed in Scilit:
- MMDocBench: Benchmarking Large Vision-Language Models for Fine-Grained Visual Document Understanding and Grounding2026
- Deep learning based visually rich document content understanding: a surveyArtificial Intelligence Review, 2026
- TokenPacker: Efficient Visual Projector for Multimodal LLMInternational Journal of Computer Vision, 2025
- OCRBench: on the hidden mystery of OCR in large multimodal modelsScience China Information Sciences, 2024
- A Survey on RAG with LLMsProcedia Computer Science, 2024
- On Scaling Up a Multilingual Vision and Language Model2024
- Intern VL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Tasks2024
- A survey on handwritten mathematical expression recognition: The rise of encoder-decoder and GNN modelsPattern Recognition, 2024
- Hierarchical multimodal transformers for Multipage DocVQAPattern Recognition, 2023
- The IAM-database: an English sentence database for offline handwriting recognitionInternational Journal on Document Analysis and Recognition (IJDAR), 2002