Recognition of on-line handwritten mathematical expressions using 2D stochastic context-free grammars and hidden Markov models

TLDR: In this article, a formal model for the recognition of on-line handwritten mathematical expressions using 2D stochastic context-free grammars and hidden Markov models is described.

About: This article is published in Pattern Recognition Letters.The article was published on 2014-01-01 and is currently open access. It has received 107 citations till now. The article focuses on the topics: Markov model & Hidden Markov model.

Figures

Figure 1: Mathematical expression initial segmentation by computing the connected components in the rendered image of the input sequence of strokes.

Figure 3: Spatial relations geometric features for automatic classification.

Table 1: Mathematical symbol recognition: recognition rate and relative improvement.

Figure 2: Rendering example of a certain sequence of strokes through linear interpolation and off-line feature extraction.

Figure 6: Relative improvement of hybrid mathematical symbol classification according to α weight (writer dependent).

Table 4: CROHME 2011 evaluation results [25].

Frequently asked questions

Q1. What are the contributions in "Recognition of on-line handwritten mathematical expressions using 2d stochastic context-free grammars and hidden markov models" ?

This paper describes a formal model for the recognition of on-line handwritten mathematical expressions using 2D stochastic context-free grammars and hidden Markov models. 

Q2. What are the future works in "Recognition of on-line handwritten mathematical expressions using 2d stochastic context-free grammars and hidden markov models" ?

For future work, given the statistical framework based on 2D-SCFG, the spatial relations classification should be improved. 

Q3. What are the main steps in the recognition of mathematical expressions?

There are two main steps in mathematical expression recognition: symbol segmentation and recognition and structural analysis of the recognized symbols. 

Q4. What was the initialization step used to recognize the set of segmentation hypotheses?

In the parsing process, the initialization step was done using a mathematical symbol classifier to recognize the set of segmentation hypotheses. 

Q5. What constraint is introduced in the parsing process?

a stochastic constraint that takes into account the geometric position of each region is introduced in the parsing process. 

Q6. What was the result of the hybrid recognition experiment?

Results showed that on-line recognition obtained higher recognition rate than off-linerecognition, and hybrid recognition significantly improved the classification accuracy. 

Q7. What were the results of the cross-validation strategy?

Writer dependent and writer independent experiments were carried out, and for each scenario 5 recognition experiments were performed as a result of the cross-validation strategy, and then the average results were obtained. 

Q8. What is the main reason why the evaluation of the structural analysis in mathematical expression recognition has been difficult?

Evaluation and comparison of the structural analysis in mathematical expression recognition has hitherto been difficult because: first, most of the proposals used private data, and second, there has been a lack of standard performance evaluation measures [5, 19]. 

Q9. What was the main issue for the performance of the spatial relations classifier?

A very important issue for the performance of the spatial relations symbol classifier was the way that the vertical center (centerver) was calculated. 

Q10. What is the simplest way to define the parsing table?

Following [15], this algorithm is essentially a dynamic programming method, which is based on the construction of a parsing table L. Each element in L is defined according to the following concepts. 

Q11. What are the results of the mathematical expression recognition experiment?

Table 3 shows the results of the mathematical expression recognition experiment, where several measures were used: symbol segmentation rate (SYMseg), symbol recognition rate for well segmented symbols (SYMrec), expression recognition rate (EXPrec), and global expression metrics EMERS and IMEGE. 

Q12. What were the constraints that guided the stochastic parsing process?

the productions of the grammar defined a set of syntactic and spatial constraints that guided the stochastic parsing process in order to build a complete structure of the most probable expression. 

Q13. What are the two major steps of recognition of handwritten mathematical expressions?

Handwritten mathematical expression recognition can be divided into two major steps [9]: symbol recognition and structural analysis. 

Q14. What is the corresponding value of the cx,y>i?

C } be the set of segmentation hypotheses (representing the input expression) provided after the pre-processing step, where c<x,y>i represents the segmentation hypothesis located at region defined by points <x, y> (top-left and bottom-right corners, respectively). 

Q15. what is the probability that both regions were arranged according to the spatial relation r?

∈Lke <xC,yC>l−k [C]∈Ll−k(p(A r−→ BC) e<xB ,yB>k [B] e <xC ,yC> l−k [C] pr(<xB, yB>,<xC , yC>))where a new subproblem e<x,y>l [A] is created from two subproblems of minor size e <xB ,yB> k [B] and e<xC ,yC>l−k [C] taking into account both the syntactic constraints, defined by p(A r−→ B C), and the following constraints: first, r represents the spatial relation, and second, pr(<xB, yB>,<xC , yC>) is the probability that both regions were arranged according to the spatial relation r (see Section 6).