Key Information Extraction and Recognition from Rich Text Images
Open Access
- 22 June 2024
- journal article
- research article
- Published by World Scientific Pub Co Pte Ltd in Vietnam Journal of Computer Science
- Vol. 11 (04) , 569-594
Abstract
Key information extraction and recognition from rich text images are crucial for various applications. There are two main tasks involved in this process: Line Item Recognition (LIR) and Key Information Localization and Extraction (KILE). LIR aims at identifying and interpreting data line items in a document. The essential information in each line item is then classified or extracted, a task known as KILE. A widely used approach for this problem is sequence based, which relies on the generalization of a language model and requires a significant amount of training time. We present an effective and reliable solution to the problem by using RoBERTa, a transformer model trained on a large corpus, along with the LION optimizer to improve the training process. A comprehensive evaluation was conducted on two different benchmarks, emphasizing two different languages, English and Vietnamese. Experimental results on DocILE indicate that the proposed framework significantly improves the KILE task with a 7.24% increase in accuracy compared to the baseline and also enhances the correct recognition rate at the LIR stage. On MCOCR, the method achieved a Character Error Rate (CER) of 28.6%, which is competitive with the state-of-the-art on this dataset.Keywords
Funding Information
- Viet Nam National University Ho Chi Minh City (DS2021-26-02)
This publication has 18 references indexed in Scilit:
- Scene Text Recognition with Permuted Autoregressive Sequence Models2022
- TGRNet: A Table Graph Reconstruction Network for Table Structure Recognition2021
- Detecting formal thought disorder by deep contextualized word representationsPsychiatry Research, 2021
- ViBERTgrid: A Jointly Trained Multi-modal 2D Document Representation for Key Information Extraction from Documents2021
- Sparse R-CNN: End-to-End Object Detection with Learnable Proposals2021
- FUNSD: A Dataset for Form Understanding in Noisy Scanned Documents2019
- An Invoice Reading System Using a Graph Convolutional Network2019
- Joint Extraction of Multiple Relations and Entities by Using a Hybrid Neural Network2017
- Feature Pyramid Networks for Object Detection2017
- An End-to-End Trainable Neural Network for Image-Based Sequence Recognition and Its Application to Scene Text RecognitionIEEE Transactions on Pattern Analysis and Machine Intelligence, 2016