Attention Where It Matters: Rethinking Visual Document Understanding with Selective Region Concentration
Authors:
- 1 October 2023
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 19460-19470
Abstract
We propose a novel end-to-end document understanding model called SeRum (SElective Region Understanding Model) for extracting meaningful information from document images, including document analysis, retrieval, and office automation. Unlike state-of-the-art approaches that rely on multi-stage technical schemes and are computationally expensive, SeRum converts document image understanding and recognition tasks into a local decoding process of the visual tokens of interest, using a content-aware token merge module. This mechanism enables the model to pay more attention to regions of interest generated by the query decoder, improving the model’s effectiveness and speeding up the decoding speed of the generative scheme. We also designed several pre-training tasks to enhance the understanding and local awareness of the model. Experimental results demonstrate that SeRum achieves state-of-the-art performance on document understanding tasks and competitive results on text spotting tasks. SeRum represents a substantial advancement towards enabling efficient and effective end-to-end document understanding.Keywords
All Related Versions
This publication has 37 references indexed in Scilit:
- End-to-End Document Recognition and Understanding with Dessurt2023
- OCR-Free Document Understanding Transformer2022
- Going Full-TILT Boogie on Document Understanding with Text-Image-Layout Transformer2021
- End-to-End Object Detection with Transformers2020
- Robotic Process Automation: An Overview and Comparison to Other Technology in Industry 4.02020
- ABCNet: Real-Time Scene Text Spotting With Adaptive Bezier-Curve Network2020
- A Large Chinese Text Dataset in the WildJournal of Computer Science and Technology, 2019
- Deep Residual Learning for Image Recognition2016
- Synthetic Data for Text Localisation in Natural Images2016
- A fast algorithm for bottom-up document layout analysisIEEE Transactions on Pattern Analysis and Machine Intelligence, 1997