SVIPTR: Fast and Efficient Scene Text Recognition with Vision Permutable Extractor
Open Access
- 21 October 2024
- conference paper
- Published by Association for Computing Machinery (ACM)
- p. 365-373
Abstract
Scene Text Recognition (STR) is an important and challenging upstream task for building structured information databases, which involves recognizing text within images of natural scenes. Although current state-of-the-art (SOTA) models for STR exhibit high performance, they typically suffer from low inference efficiency due to their reliance on hybrid architectures comprised of visual encoders and sequence decoders. In this work, we propose a VIsion Permutable extractor for fast and efficient Scene Text Recognition (SVIPTR), which achieves an impressive balance between high performance and rapid inference speeds in the domain of STR. Specifically, SVIPTR leverages a visual-semantic extractor with a pyramid structure, characterized by the Permutation and combination of local and global self-attention layers. This design results in a lightweight and efficient model and its inference is insensitive to input length. Extensive experimental results on various standard datasets for both Chinese and English scene text recognition validate the superiority of SVIPTR. Notably, the SVIPTR-T (Tiny) variant delivers highly competitive accuracy on par with other lightweight models and achieves SOTA inference speeds. Meanwhile, the SVIPTR-L (Large) attains SOTA accuracy in single-encoder-type models, while maintaining a low parameter count and favorable inference speed. Our proposed method provides a compelling solution for the STR challenge, which greatly benefits real-world applications requiring fast and efficient STR. The code is publicly available at https://github.com/cxfyxl/VIPTR.Keywords
All Related Versions
This publication has 20 references indexed in Scilit:
- Multi-granularity Prediction for Scene Text Recognition2022
- CSWin Transformer: A General Vision Transformer Backbone with Cross-Shaped Windows2022
- Swin Transformer: Hierarchical Vision Transformer using Shifted Windows2021
- Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutions2021
- From Two to One: A New Scene Text Recognizer with Visual Language Modeling Network2021
- Vision Transformer for Fast and Efficient Scene Text Recognition2021
- On Recognizing Texts of Arbitrary Shapes with 2D Self-Attention2020
- Synthetic Data for Text Localisation in Natural Images2016
- Reading Text in the Wild with Convolutional Neural NetworksInternational Journal of Computer Vision, 2015
- End-to-end scene text recognition2011