Francisco Alvaro

There are thousands of Unicode characters and hence it can be hard to visually find a particular one. For this reason, we aimed at developing a tool that allows to handwrite a character and receive a list of the most similar candidates to that input. This tool will be integrated in a math editor which handles more than 5,000 different Unicode characters. Since no public datasets were found to fit our needs, we crowdsourced the acquisition of online handwritten data for training purposes. We developed a neural network combining convolutional layers with shape-based features to classify online handwritten Unicode characters. To make the model more robust to input variability, we used data augmentation in the form of affine transformations. We achieved a top-20 error rate of 12.64% on validation data and received positive feedback from users, thus validating that crowdsourcing is a proper method for online handwriting acquisition. Finally, we deployed the model wrapped in a JSON-based REST API and released a public demo using it. This way, we present the full development cycle of a Unicode character classifier.

In mathematical expression recognition, symbol classification is a crucial step. Numerous approaches for recognizing handwritten math symbols have been published, but most of them are either an online approach or a hybrid approach. There is an absence of a study focused on offline features for handwritten math symbol recognition. Furthermore, many papers provide results difficult to compare. In this paper we assess the performance of several well-known offline features for this task. We also test a novel set of features based on polar histograms and the vertical repositioning method for feature extraction. Finally, we report and analyze the results of several experiments using recurrent neural networks on a large public database of online handwritten math expressions. The combination of online and offline features significantly improved the recognition rate.

Recognition of on-line handwritten mathematical symbols has been tackled using different methods, but the recognition rates achieved until now still leave room for improvement. Many of the published approaches are based on hidden Markov models, and some of them use off-line information extracted from the on-line data. In this paper, we present a set of hybrid features that combine both on-line and off-line information. Lately, recurrent neural networks have demonstrated to obtain good results and they have outperformed hidden Markov models in several sequence learning tasks, including handwritten text recognition. Hence, we also studied a state-of-the-art recurrent neural network classifier and we compared its performance with a classifier based on hidden Markov models. Experiments using a large public database showed that both the new proposed features and recurrent neural network classifier improved significantly the classification results.