TIMTQE: Benchmarking Machine Translation Quality Estimation for Text Images

Authors:
Publisher website
Google Scholar
Abstract
Accurate quality estimation is critical for the reliable machine translation of text from historical document images. However, existing quality estimation methods typically presuppose clean source text, failing to account for the visual noise and optical character recognition errors prevalent in authentic documents. This challenge is compounded by the lack of standardized benchmarks. To address these issues, we introduce TIMTQE, a novel multimodal and multilingual benchmark for quality estimation on text images. TIMTQE comprises a human-annotated corpus of historical documents and a large-scale synthetic dataset, designed for reference-free quality prediction directly from noisy images. We evaluate modern multimodal large language models and demonstrate that end-to-end models achieve substantially greater robustness on high-noise images compared to conventional cascaded systems that rely on separate optical character recognition and quality estimation modules.
Funding Information
  • National Natural Science Foundation of China (62503015)

This publication has 16 references indexed in Scilit: