Online Handwritten Mathematical Expression Recognition and Applications: A Survey
Open Access
- 2 March 2021
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 9, 38352-38373
Abstract
Handwritten mathematical expressions are an essential part of many domains, including education, engineering, and science. The pervasive availability of computationally powerful touch-screen devices, similar to the recent emergence of deep neural networks as high-quality sequence recognition models, result in the widespread adoption of online recognition of handwritten mathematical expressions. Also, a deeper study and improvement of such technologies is necessary to address the current challenges posed by the extensive usage of distance learning, and remote work due to the world pandemic. This paper delineates the state-of-the-art recognition methods along with the user’s experience in pen-centric applications for operating with handwritten mathematical expressions. Recognition methods have been categorized into classes, with a description of their merits and limitations. Particular attention is paid to end-to-end approaches based on encoder-decoder architecture and multi-modal input. Evaluation protocols and open benchmark datasets are considered as well as the comparison of the recognition performance, based on open competition results. The use of handwritten math recognition is illustrated by examples of applications for various fields and platforms. A distinctive part of the survey is that we also considered how UI design relies on the use of different recognition approaches, which is aimed at helping potential researchers improve the performance of the introduced approaches toward the best responses in practical applications. Finally, this paper presents the prospective survey of future research directions in handwritten mathematical expression recognition and their applications.Keywords
This publication has 121 references indexed in Scilit:
- Recognition and retrieval of mathematical expressionsInternational Journal on Document Analysis and Recognition (IJDAR), 2011
- EMERS: a tree matching–based performance evaluation of mathematical expression recognition systemsInternational Journal on Document Analysis and Recognition (IJDAR), 2010
- An evaluation of stylus-based text entry methods on handheld devices studied in different user mobility statesPervasive and Mobile Computing, 2009
- Efficient search strategy in structural analysis for handwritten mathematical expression recognitionPattern Recognition, 2008
- Newton's Pen: A pen-based tutoring system for staticsComputers & Graphics, 2008
- Recognizing mathematical expressions using tree transformationIEEE Transactions on Pattern Analysis and Machine Intelligence, 2002
- Error detection, error correction and performance evaluation in on-line mathematical expression recognitionPattern Recognition, 2001
- Towards an art based mathematical editor, that uses on-line handwritten symbol recognitionPattern Recognition, 1995
- An Experimental Implementation of a Document Recognition System for Papers Containing Mathematical Expressions1992
- A method for the structural analysis of two-dimensional mathematical expressionsInformation Sciences, 1970