HTSR-Pollen: Handwritten Text Synthesis and Recognition System to Overcome Data Scarcity
Open Access
- 7 April 2026
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 14, 54395-54413
Abstract
Offline Handwritten Text Recognition (HTR) systems recognize and transcribe handwritten text from scanned images into digital formats. The field has become important due to the need for document digitization and data entry automation in various industries. Accurate recognition requires large and varied datasets for training optical models, where collecting and labeling these datasets is often time-consuming and impractical. To address this challenge, data augmentation and transfer learning are commonly used. Nevertheless, these traditional methods may lead to overfitting and performance degradation when data are scarce. This work proposes integrating Conditional Generative Adversarial Networks (CGANs) for data synthesis into optical model training to improve handwriting recognition in data-scarce scenarios. To validate our proposal, we conducted a study that included: (i) an exploration to establish an optimal configuration for traditional data augmentation; and (ii) extensive experiments using seven datasets. In addition, these datasets were partitioned into training subsets to simulate diverse data-scarcity conditions. Averaged over all subsets and optical models, data synthesis achieved the highest reductions compared to the baseline trained from scratch without augmentation. It reduced the Character Error Rate (CER) by 41.1% and the Word Error Rate (WER) by 28.1%. Transfer learning achieved reductions of 34.4% in CER and 23.5% in WER. Lastly, traditional data augmentation achieved reductions of 13.8% in CER and 12.3% in WER. These findings highlight the importance of data synthesis for improving HTR systems, particularly in data-scarcity contexts.Keywords
Funding Information
- Conselho Nacional de Desenvolvimento Científico e Tecnológico (142208/2024-8)
- Fundação de Amparo à Ciência e Tecnologia do Estado de Pernambuco (APQ-1216-1.03/22)
- Coordenação de Aperfeiçoamento de Pessoal de Nível Superior (Finance Code 001)
This publication has 58 references indexed in Scilit:
- ICDAR 2024 Competition on Handwritten Text Recognition in Brazilian Essays – BRESSAY2024
- BRESSAY: A Brazilian Portuguese Dataset for Offline Handwritten Text Recognition2024
- Handwritten Text Generation with Character-Specific Encoding for Style Imitation2023
- WordStylist: Styled Verbatim Handwritten Text Generation with Latent Diffusion Models2023
- Active Transfer Learning for Handwriting Recognition2022
- Script-Level Word Sample Augmentation for Few-Shot Handwritten Text Recognition2022
- A Light Transformer-Based Architecture for Handwritten Text Recognition2022
- Rescoring Sequence-to-Sequence Models for Text Line Recognition with CTC-Prefixes2022
- SPAN: A Simple Predict & Align Network for Handwritten Paragraph Recognition2021
- GANwriting: Content-Conditioned Generation of Styled Handwritten Word Images2020