Granularity-Aware Single-Point Scene Text Spotting With Sequential Recurrence Self-Attention
- 22 July 2024
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Circuits and Systems for Video Technology
- Vol. 34 (12) , 12524-12534
Abstract
Scene text spotting, a unified framework between text detection and text recognition, has made great progress in recent years. Existing methods usually adopt the fully-supervised learning strategy, which relies on time-consuming location annotations, particularly for scene texts with arbitrary shapes. In this paper, we propose a weakly-supervised scene text spotting method via the location labels of single points with the corresponding text transcriptions. Due to the weak location annotations for challenging scene texts, previous weakly-supervised methods adopting the convolution neural network structure make it hard to model the different-scale text feature representations under blurring or nosing scenarios. In addition, as the single-point location can only cover part of the text instance, it will burden the confusion of sequential-like scene text recognition. To address these issues, we present a novel sequential recurrence self-attention for granularity-aware single-point scene text spotting. Specifically, we first enhance the scene text feature representations with different scales by integrating the global intra-interaction of high-level features with the low-level local features. Then, based on the granularity-aware text features, we decode them into text transcriptions in the sequential recurrence self-attention manner to capture the sequence-dependent relation in character-level semantics and locations. Extensive experiments show that our proposed method outperforms existing state-of-the-art weakly-supervised scene text spotters by a large margin.Keywords
Funding Information
- National Key Research and Development Program of China (2022YFB3103500)
- National Natural Science Foundation of China (62302532)
- Shenzhen Science and Technology Program (KQTD20221101093559018, 202206193000001, 20220816225523001)
- Fundamental Research Funds for the Central Universities, Sun Yat-sen University (24qnpy152)
This publication has 58 references indexed in Scilit:
- GLASS: Global to Local Attention for Scene-Text Spotting2022
- Mask TextSpotter v3: Segmentation Proposal Network for Robust Scene Text Spotting2020
- AE TextSpotter: Learning Visual and Linguistic Representation for Ambiguous Text Spotting2020
- End-to-End Object Detection with Transformers2020
- Scene Text Detection and Recognition: The Deep Learning EraInternational Journal of Computer Vision, 2020
- Curved scene text detection via transverse and longitudinal sequence connectionPattern Recognition, 2019
- Mask TextSpotter: An End-to-End Trainable Neural Network for Spotting Text with Arbitrary Shapes2018
- An End-to-End TextSpotter with Explicit Alignment and Attention2018
- Feature Pyramid Networks for Object Detection2017
- Deep Residual Learning for Image Recognition2016