Pixel Adapter: A Graph-Based Post-Processing Approach for Scene Text Image Super-Resolution
Open Access
- 26 October 2023
- conference paper
- Published by Association for Computing Machinery (ACM)
- p. 2168-2179
Abstract
Current Scene text image super-resolution approaches primarily focus on extracting robust features, acquiring text information, and complex training strategies to generate super-resolution images. However, the upsampling module, which is crucial in the process of converting low-resolution images to high-resolution ones, has received little attention in existing works. To address this issue, we propose the Pixel Adapter Module (PAM) based on graph attention to address pixel distortion caused by upsampling. The PAM effectively captures local structural information by allowing each pixel to interact with its neighbors and update features. Unlike previous graph attention mechanisms, our approach achieves 2-3 orders of magnitude improvement in efficiency and memory utilization by eliminating the dependency on sparse adjacency matrices and introducing a sliding window approach for efficient parallel computation. Additionally, we introduce the MLP-based Sequential Residual Block (MSRB) for robust feature extraction from text images, and a Local Contour Awareness loss (ℒlca) to enhance the model's perception of details. Comprehensive experiments on TextZoom demonstrate that our proposed method generates high-quality super-resolution images, surpassing existing methods in recognition accuracy. For single-stage and multi-stage strategies, we achieved improvements of 0.7% and 2.6%, respectively, increasing the performance from 52.6% and 53.7% to 53.3% and 56.3%. The code is available at https://github.com/wenyu1009/RTSRN.Keywords
All Related Versions
Funding Information
- National Key Research and Development Program of China (2020YFB2103803.)
- Supercomputing Center of University of Science and Technology of China
This publication has 23 references indexed in Scilit:
- Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network2017
- An End-to-End Trainable Neural Network for Image-Based Sequence Recognition and Its Application to Scene Text RecognitionIEEE Transactions on Pattern Analysis and Machine Intelligence, 2016
- Synthetic Data for Text Localisation in Natural Images2016
- Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network2016
- Reading Text in the Wild with Convolutional Neural NetworksInternational Journal of Computer Vision, 2015
- Recognizing Text with Perspective Distortion in Natural Scenes2013
- Gradient Profile Prior and Its Applications in Image Super-Resolution and EnhancementIEEE Transactions on Image Processing, 2010
- Histograms of Oriented Gradients for Human Detection2005
- An automatic road sign recognition system based on a computational model of human recognition processingComputer Vision and Image Understanding, 2004
- Image quality assessment: from error visibility to structural similarityIEEE Transactions on Image Processing, 2004