Daniil V. Tropin

Document image rectification problem is crucial in document analysis. Most of the current state-of-the-art methods addressing it are data-driven and rely on neural network approaches. However, despite satisfactory rectifications, such methods’ time performance is poor, making them unsuitable for mobile on-device acquisition. The present work concentrates on a specific (but common) case of document physical distortion – the documents with a single crease.We investigate the properties of a surface comprised of two planes captured by a pinhole camera. Namely, we provide the methods to obtain the transformation between such an image and the template image having successfully localized the document in a frame. It can be utilized in on-device recognition systems: it takes only 3 ms to estimate transformation parameters and about a quarter of a second to rectify an image on a smartphone CPU. We propose a novel dataset FDI-AC containing 200 real images of documents with a single crease in different positions. We conduct experiments comparing our approach with current state-of-the-art setting a baseline performance on FDI-AC. These experiments show that the proposed algorithm outperforms image rectification transformer network GeoTr in rectification accuracy and time performance.

Presentation of folded documents is not an uncommon case in modern society. Digitizing such documents by capturing them with a smartphone camera can be tricky since a crease can divide the document contents into separate planes. To unfold the document, one could hold the edges potentially obscuring it in a captured image. While there are many geometrical rectification methods, they were usually developed for arbitrary bends and folds. We consider such algorithms and propose a novel approach Unfolder developed specifically for images of documents with a crease from folding in half. Unfolder is robust to projective distortions of the document image and does not fragment the image in the vicinity of a crease after rectification. A new Folded Document Images dataset was created to investigate the rectification accuracy of folded (2, 3, 4, and 8 folds) documents. The dataset includes 1600 images captured when document placed on a table and when held in hand. The Unfolder algorithm allowed for a recognition error rate of 0.33, which is better than the advanced neural network methods DocTr (0.44) and DewarpNet (0.57). The average runtime for Unfolder was only 0.25 s/image on an iPhone XR.

The article is devoted to approaches and methods for analyzing document images, which were developed and used by scientists of the scientific school of V.L. Arlazarov to solve problems of the type definition and localization of documents with a known structure. It describes the principles for building solutions that have emerged as input data have become more complex and performance requirements have stricted. The methods presented in the article demonstrate the scientific path of the school from working with scanned images to photographs and video stream frames, from the most general classes of documents tied to text structure to the strictest ones based on their visual features.

The demand for on-device document recognition systems increases in conjunction with the emergence of more strict privacy and security requirements. In such systems, there is no data transfer from the end device to a third-party information processing servers. The response time is vital to the user experience of on-device document recognition. Combined with the unavailability of discrete GPUs, powerful CPUs, or a large RAM capacity on consumer-grade end devices such as smartphones, the time limitations put significant constraints on the computational complexity of the applied algorithms for on-device execution. In this work, we consider document location in an image without prior knowledge of the document content or its internal structure. In accordance with the published works, at least 5 systems offer solutions for on-device document location. All these systems use a location method which can be considered Hough-based. The precision of such systems seems to be lower than that of the state-of-the-art solutions which were not designed to account for the limited computational resources. We propose an advanced Hough-based method. In contrast with other approaches, it accounts for the geometric invariants of the central projection model and combines both edge and color features for document boundary detection. The proposed method allowed for the second best result for SmartDoc dataset in terms of precision, surpassed by U-net like neural network. When evaluated on a more challenging MIDV-500 dataset, the proposed algorithm guaranteed the best precision compared to published methods. Our method retained the applicability to on-device computations.

In this work, we consider a problem of quadrilateral document borders detection in images captured by a mobile device’s camera. State-of-the-art algorithms for the quadrilateral document borders detection are not designed for cases when one of the document borders is either completely out of the frame, obscured, or of low contrast. We propose the algorithm which correctly processes the image in such cases. It is built on the classical contour-based algorithm. We modify the latter using the document’s aspect ratio which is known a priori. We demonstrate that this modification reduces the number of incorrect detections by 34% on an open dataset MIDV-500.