Pixel Adapter: A Graph-Based Post-Processing Approach for Scene Text Image Super-Resolution
Authors:
- Wenyu ZhangWZWenyu Zhang
- Xin DengXDXin Deng
- Baojun JiaBJBaojun Jia
- Xingtong YuXYXingtong Yu
- Yifan ChenYCYifan Chen
- Jin MaJMJin Ma
- Qing DingQDQing Ding
- Xinming ZhangXinming Zhang
Preprint
Abstract
Current Scene text image super-resolution approaches primarily focus on extracting robust features, acquiring text information, and complex training strategies to generate super-resolution images. However, the upsampling module, which is crucial in the process of converting low-resolution images to high-resolution ones, has received little attention in existing works. To address this issue, we propose the Pixel Adapter Module (PAM) based on graph attention to address pixel distortion caused by upsampling. The PAM effectively captures local structural information by allowing each pixel to interact with its neighbors and update features. Unlike previous graph attention mechanisms, our approach achieves 2-3 orders of magnitude improvement in efficiency and memory utilization by eliminating the dependency on sparse adjacency matrices and introducing a sliding window approach for efficient parallel computation. Additionally, we introduce the MLP-based Sequential Residual Block (MSRB) for robust feature extraction from text images, and a Local Contour Awareness loss ($\mathcal{L}_{lca}$) to enhance the model's perception of details. Comprehensive experiments on TextZoom demonstrate that our proposed method generates high-quality super-resolution images, surpassing existing methods in recognition accuracy. For single-stage and multi-stage strategies, we achieved improvements of 0.7\% and 2.6\%, respectively, increasing the performance from 52.6\% and 53.7\% to 53.3\% and 56.3\%. The code is available at https://github.com/wenyu1009/RTSRN.Keywords
All Related Versions
This publication has 12 references indexed in Scilit:
- Text Prior Guided Scene Text Image Super-ResolutionIEEE Transactions on Image Processing, 2023
- Street View Text Recognition With Deep Learning for Urban Scene Understanding in Intelligent Transportation SystemsIEEE Transactions on Intelligent Transportation Systems, 2020
- Toward Real-World Single Image Super-Resolution: A New Benchmark and a New Model2019
- A set of benchmarks for Handwritten Text Recognition on historical documentsPattern Recognition, 2019
- MORAN: A Multi-Object Rectified Attention Network for scene text recognitionPattern Recognition, 2019
- ASTER: An Attentional Scene Text Recognizer with Flexible RectificationIEEE Transactions on Pattern Analysis and Machine Intelligence, 2018
- An End-to-End Trainable Neural Network for Image-Based Sequence Recognition and Its Application to Scene Text RecognitionIEEE Transactions on Pattern Analysis and Machine Intelligence, 2016
- Reading Text in the Wild with Convolutional Neural NetworksInternational Journal of Computer Vision, 2015
- Gradient Profile Prior and Its Applications in Image Super-Resolution and EnhancementIEEE Transactions on Image Processing, 2010
- Image quality assessment: from error visibility to structural similarityIEEE Transactions on Image Processing, 2004