Biao Yang
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Biao Yang has a total of 22 co-authors. Here are the top co-authors by number of shared publications:
- Xiang Bai
Huazhong University of Science and Technology, Wuhan, China
6 shared publications - Mingkun Yang
Huazhong University of Science and Technology
4 shared publications - Yuliang Liu
Huazhong University of Science and Technology, Wuhan, China
4 shared publications - Yingying Zhu
School of Electronic Information and Communications, Huazhong University of Science and Technology, Wuhan, China
3 shared publications - Chenyang Gao
Huazhong University of Science and Technology
2 shared publications - Minghui Liao
Huawei Inc
2 shared publications
Latest Publications
- Top Cited Papers
- 1 May 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Pattern Analysis and Machine Intelligence
- Vol. 48 (5) , 6008-6019
TextMonkey: An OCR-Free Large Multimodal Model for Understanding Document
We present TextMonkey, a large multimodal model (LMM) tailored for text-centric tasks. Our approach introduces enhancement across several dimensions: By adopting Shifted Window Attention layer, we achieve cross-window connectivity at higher input resolutions and stabilize early training; We hypothesize that images may contain redundant tokens, and by using similarity to filter out significant tokens, we can not only streamline the token length but also enhance the model’s performance. Moreover, by expanding our model’s capabilities to encompass text spotting and grounding, and incorporating positional information into responses, we enhance interpretability. Evaluation on 12 benchmarks shows notable improvements: 5.2% in Scene Text-Centric tasks (including STVQA, TextVQA, and OCRVQA), 6.9% in Document-Oriented tasks (such as DocVQA, InfoVQA, ChartVQA, DeepForm, Kleister Charity, and WikiTableQuestions), and 2.8% in Key Information Extraction tasks (comprising FUNSD, SROIE, and POIE). It outperforms in scene text spotting with a 10.9% increase and sets a new standard on OCRBench, a comprehensive benchmark consisting of 29 OCR-related assessments, with a score of 561, surpassing previous open-sourced large multimodal models for document understanding.
- 1 May 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Pattern Analysis and Machine Intelligence
- Vol. 48 (5) , 6008-6019
Top Cited PapersTextMonkey: An OCR-Free Large Multimodal Model for Understanding Document
We present TextMonkey, a large multimodal model (LMM) tailored for text-centric tasks. Our approach introduces enhancement across several dimensions: By adopting Shifted Window Attention layer, we achieve cross-window connectivity at higher input resolutions and stabilize early training; We hypothesize that images may contain redundant tokens, and by using similarity to filter out significant tokens, we can not only streamline the token length but also enhance the model’s performance. Moreover, by expanding our model’s capabilities to encompass text spotting and grounding, and incorporating positional information into responses, we enhance interpretability. Evaluation on 12 benchmarks shows notable improvements: 5.2% in Scene Text-Centric tasks (including STVQA, TextVQA, and OCRVQA), 6.9% in Document-Oriented tasks (such as DocVQA, InfoVQA, ChartVQA, DeepForm, Kleister Charity, and WikiTableQuestions), and 2.8% in Key Information Extraction tasks (comprising FUNSD, SROIE, and POIE). It outperforms in scene text spotting with a 10.9% increase and sets a new standard on OCRBench, a comprehensive benchmark consisting of 29 OCR-related assessments, with a score of 561, surpassing previous open-sourced large multimodal models for document understanding.
- 9 September 2024
- Published by Springer Nature
- p. 245-262
Knowledge Mining of Scene Text for Referring Expression Comprehension
- 9 September 2024
- Published by Springer Nature
- p. 245-262
Knowledge Mining of Scene Text for Referring Expression Comprehension
- Open Access
Class-Aware Mask-Guided Feature Refinement for Scene Text Recognition
Scene text recognition is a rapidly developing field that faces numerous challenges due to the complexity and diversity of scene text, including complex backgrounds, diverse fonts, flexible arrangements, and accidental occlusions. In this paper, we propose a novel approach called Class-Aware Mask-guided feature refinement (CAM) to address these challenges. Our approach introduces canonical class-aware glyph masks generated from a standard font to effectively suppress background and text style noise, thereby enhancing feature discrimination. Additionally, we design a feature alignment and fusion module to incorporate the canonical mask guidance for further feature refinement for text recognition. By enhancing the alignment between the canonical mask feature and the text feature, the module ensures more effective fusion, ultimately leading to improved recognition performance. We first evaluate CAM on six standard text recognition benchmarks to demonstrate its effectiveness. Furthermore, CAM exhibits superiority over the state-of-the-art method by an average performance gain of 4.1% across six more challenging datasets, despite utilizing a smaller model size. Our study highlights the importance of incorporating canonical mask guidance and aligned feature refinement techniques for robust scene text recognition. The code is available at https://github.com/MelosY/CAM.
Open AccessClass-Aware Mask-Guided Feature Refinement for Scene Text Recognition
Scene text recognition is a rapidly developing field that faces numerous challenges due to the complexity and diversity of scene text, including complex backgrounds, diverse fonts, flexible arrangements, and accidental occlusions. In this paper, we propose a novel approach called Class-Aware Mask-guided feature refinement (CAM) to address these challenges. Our approach introduces canonical class-aware glyph masks generated from a standard font to effectively suppress background and text style noise, thereby enhancing feature discrimination. Additionally, we design a feature alignment and fusion module to incorporate the canonical mask guidance for further feature refinement for text recognition. By enhancing the alignment between the canonical mask feature and the text feature, the module ensures more effective fusion, ultimately leading to improved recognition performance. We first evaluate CAM on six standard text recognition benchmarks to demonstrate its effectiveness. Furthermore, CAM exhibits superiority over the state-of-the-art method by an average performance gain of 4.1% across six more challenging datasets, despite utilizing a smaller model size. Our study highlights the importance of incorporating canonical mask guidance and aligned feature refinement techniques for robust scene text recognition. The code is available at https://github.com/MelosY/CAM.
- 11 January 2024
- Published by Elsevier in Pattern Recognition Letters
- Vol. 178, 174-180
Sequential visual and semantic consistency for semi-supervised text recognition
- 11 January 2024
- Published by Elsevier in Pattern Recognition Letters
- Vol. 178, 174-180
Sequential visual and semantic consistency for semi-supervised text recognition
- 31 December 2023
- Published by Elsevier in Pattern Recognition
- Vol. 149
Class-Aware Mask-guided feature refinement for scene text recognition
- 31 December 2023
- Published by Elsevier in Pattern Recognition
- Vol. 149
Class-Aware Mask-guided feature refinement for scene text recognition
- 19 August 2023
- Published by Springer Nature
- p. 587-600
ICDAR 2023 Competition on Detecting Tampered Text in Images
- 19 August 2023
- Published by Springer Nature
- p. 587-600
ICDAR 2023 Competition on Detecting Tampered Text in Images
- 19 August 2023
- Published by Springer Nature
- p. 402-420
TextREC: A Dataset for Referring Expression Comprehension with Reading Comprehension
- 19 August 2023
- Published by Springer Nature
- p. 402-420
TextREC: A Dataset for Referring Expression Comprehension with Reading Comprehension