How to Choose Pretrained Handwriting Recognition Models for Single Writer Fine-Tuning
Authors:
- Vittorio PippiVPVittorio Pippi
- Silvia CascianelliSCSilvia Cascianelli
- Christopher KermorvantCKChristopher Kermorvant
- Rita CucchiaraRCRita Cucchiara
Preprint
Abstract
Recent advancements in Deep Learning-based Handwritten Text Recognition (HTR) have led to models with remarkable performance on both modern and historical manuscripts in large benchmark datasets. Nonetheless, those models struggle to obtain the same performance when applied to manuscripts with peculiar characteristics, such as language, paper support, ink, and author handwriting. This issue is very relevant for valuable but small collections of documents preserved in historical archives, for which obtaining sufficient annotated training data is costly or, in some cases, unfeasible. To overcome this challenge, a possible solution is to pretrain HTR models on large datasets and then fine-tune them on small single-author collections. In this paper, we take into account large, real benchmark datasets and synthetic ones obtained with a styled Handwritten Text Generation model. Through extensive experimental analysis, also considering the amount of fine-tuning lines, we give a quantitative indication of the most relevant characteristics of such data for obtaining an HTR model able to effectively transcribe manuscripts in small collections with as little as five real fine-tuning lines.Keywords
All Related Versions
This publication has 48 references indexed in Scilit:
- A Comprehensive Comparison of Open-Source Libraries for Handwritten Text Recognition in Norwegian2022
- Rescoring Sequence-to-Sequence Models for Text Line Recognition with CTC-Prefixes2022
- Boosting modern and historical handwritten text recognition with deformable convolutionsInternational Journal on Document Analysis and Recognition (IJDAR), 2022
- Learning to Read L’Infinito: Handwritten Text Recognition with Synthetic Training Data2021
- SmartPatch: Improving Handwritten Word Imitation with Patch Discriminators2021
- Transformer for Handwritten Text Recognition Using Bidirectional Post-decoding2021
- A Database and Visualization of the Similarity of Contemporary Lexicons2021
- GANwriting: Content-Conditioned Generation of Styled Handwritten Word Images2020
- Start, Follow, Read: End-to-End Full-Page Handwriting Recognition2018
- Combining shape and physical modelsfor online cursive handwriting synthesisInternational Journal on Document Analysis and Recognition (IJDAR), 2005