Document recognition and retrieval technologies complement one another, providing improved access to increasingly large document collections. While recognition and retrieval of textual information is fairly mature, with wide-spread availability of optical character recognition and text-based search engines, recognition and retrieval of graphics such as images, figures, tables, diagrams, and mathematical expressions are in comparatively early stages of research. This paper surveys the state of the art in recognition and retrieval of mathematical expressions, organized around four key problems in math retrieval (query construction, normalization, indexing, and relevance feedback), and four key problems in math recognition (detecting expressions, detecting and classifying symbols, analyzing symbol layout, and constructing a representation of meaning). Of special interest is the machine learning problem of jointly optimizing the component algorithms in a math recognition system, and developing effective indexing, retrieval and relevance feedback algorithms for math retrieval. Another important open problem is developing user interfaces that seamlessly integrate recognition and retrieval. Activity in these important research areas is increasing, in part because math notation provides an excellent domain for studying problems common to many document and graphics recognition and retrieval applications, and also because mature applications will likely provide substantial benefits for education, research, and mathematical literacy.

/pdf/recognition-and-retrieval-of-mathematical-expressions-2x3ms5stvn.pdf

Recognition and retrieval of mathematical expressions

We present a systematic study on the recognition of 3D gestures using spatially convenient input devices. Specifically, we examine the linear acceleration-sensing Nintendo Wii Remote coupled with the angular velocity-sensing Nintendo Wii MotionPlus. For the study, we created a 3D gesture database, collecting data on 25 distinct gestures totalling 8500 gestures samples. Our experiment explores how the number of gestures and the amount of gestures samples used to train two commonly used machine learning algorithms, a linear and AdaBoost classifier, affect overall recognition accuracy. We examined these gesture recognition algorithms with user dependent and user independent training approaches and explored the affect of using the Wii Remote with and without the Wii MotionPlus attachment. Our results show that in the user dependent case, both the Ad-aBoost and linear classification algorithms can recognize up to 25 gestures at over 90% accuracy, with 15 training samples per gesture, and up to 20 gestures at over 90% accuracy, with only five training samples per gesture. In particular, all 25 gestures could be recognized at over 99% accuracy with the linear classifier using 15 training samples per gesture, with the Wii Remote coupled with the Wii MotionPlus. In addition, both algorithms can recognize up to nine gestures at over 90% accuracy using a user independent training database with 100 samples per gesture. The Wii MotionPlus attachment played a significant role in improving accuracy in both the user dependent and independent cases.

/pdf/breaking-the-status-quo-improving-3d-gesture-recognition-162rpc3kid.pdf

Breaking the status quo: Improving 3D gesture recognition with spatially convenient input devices

Iconic and multi-stroke gesture recognition

https://hal.archives-ouvertes.fr/hal-00933509/document

HBF49 feature set: A first unified baseline for online symbol recognition

This paper presents a new robust method for inertial MEM (MicroElectroMechanical systems) 3D gesture recognition. The linear acceleration and the angular velocity, respectively provided by the accelerometer and the gyrometer, are sampled in time resulting in 6D values at each time step which are used as inputs for the gesture recognition system. We propose to build a system based on Bidirectional Long Short-Term Memory Recurrent Neural Networks (BLSTM-RNN) for gesture classification from raw MEM data. We also compare this system to a geometric approach using DTW (Dynamic Time Warping) and a statistical method based on HMM (Hidden Markov Model) from filtered and denoised MEM data. Experimental results on 22 individuals producing 14 gestures in the air show that the proposed approach outperforms classical classification methods with a classification mean rate of 95.57% and a standard deviation of 0.50 for 616 test gestures. Furthermore, these experiments underline that combining accelerometer and gyrometer information gives better results that using a single inertial description.

/pdf/blstm-rnn-based-3d-gesture-classification-4nllv5ulqw.pdf

BLSTM-RNN Based 3D Gesture Classification

We present a practical technique for using a writer-independent recognition engine to improve the accuracy and speed while reducing the training requirements of a writer-dependent symbol recognizer. Our writer-dependent recognizer uses a set of binary classifiers based on the AdaBoost learning algorithm, one for each possible pairwise symbol comparison. Each classifier consists of a set of weak learners, one of which is based on a writer-independent handwriting recognizer. During online recognition, we also use the n-best list of the writer-independent recognizer to prune the set of possible symbols and, thus, reduce the number of required binary classifications. In this paper, we describe the geometric and statistical features used in our recognizer and our all-pairs classification algorithm. We also present the results of experiments that quantify the effect incorporating a writer-independent recognition engine into a writer-dependent recognizer has on accuracy, speed, and user training time.

R.C. Zeleznik

Papers

A Practical Approach for Writer-Dependent Symbol Recognition Using a Writer-Independent Symbol Recognizer