Bridging Synthetic and Real Worlds for Pre-training Scene Text Detectors
Authors:
Preprint
Abstract
Existing scene text detection methods typically rely on extensive real data for training. Due to the lack of annotated real images, recent works have attempted to exploit large-scale labeled synthetic data (LSD) for pre-training text detectors. However, a synth-to-real domain gap emerges, further limiting the performance of text detectors. Differently, in this work, we propose \textbf{FreeReal}, a real-domain-aligned pre-training paradigm that enables the complementary strengths of both LSD and unlabeled real data (URD). Specifically, to bridge real and synthetic worlds for pre-training, a novel glyph-based mixing mechanism (GlyphMix) is tailored for text images. GlyphMix delineates the character structures of synthetic images and embeds them as graffiti-like units onto real images. Without introducing real domain drift, GlyphMix freely yields real-world images with annotations derived from synthetic labels. Furthermore, when given free fine-grained synthetic labels, GlyphMix can effectively bridge the linguistic domain gap stemming from English-dominated LSD to URD in various languages. Without bells and whistles, FreeReal achieves average gains of 4.56\%, 3.85\%, 3.90\%, and 1.97\% in improving the performance of DBNet, PANet, PSENet, and FCENet methods, respectively, consistently outperforming previous pre-training methods by a substantial margin across four public datasets. Code will be released soon.Keywords
All Related Versions
This publication has 42 references indexed in Scilit:
- Language Matters: A Weakly Supervised Vision-Language Pre-training Approach for Scene Text Detection and Spotting2022
- Total-Text: toward orientation robustness in scene text detectionInternational Journal on Document Analysis and Recognition (IJDAR), 2019
- Arbitrary Shape Scene Text Detection With Adaptive Text Region Representation2019
- Shape Robust Text Detection With Progressive Scale Expansion Network2019
- Character Region Awareness for Text Detection2019
- Curved scene text detection via transverse and longitudinal sequence connectionPattern Recognition, 2019
- TextSnake: A Flexible Representation for Detecting Text of Arbitrary Shapes2018
- Detecting Oriented Text in Natural Images by Linking Segments2017
- Synthetic Data for Text Localisation in Natural Images2016
- Detecting texts of arbitrary orientations in natural images2012