Document recognition and retrieval technologies complement one another, providing improved access to increasingly large document collections. While recognition and retrieval of textual information is fairly mature, with wide-spread availability of optical character recognition and text-based search engines, recognition and retrieval of graphics such as images, figures, tables, diagrams, and mathematical expressions are in comparatively early stages of research. This paper surveys the state of the art in recognition and retrieval of mathematical expressions, organized around four key problems in math retrieval (query construction, normalization, indexing, and relevance feedback), and four key problems in math recognition (detecting expressions, detecting and classifying symbols, analyzing symbol layout, and constructing a representation of meaning). Of special interest is the machine learning problem of jointly optimizing the component algorithms in a math recognition system, and developing effective indexing, retrieval and relevance feedback algorithms for math retrieval. Another important open problem is developing user interfaces that seamlessly integrate recognition and retrieval. Activity in these important research areas is increasing, in part because math notation provides an excellent domain for studying problems common to many document and graphics recognition and retrieval applications, and also because mature applications will likely provide substantial benefits for education, research, and mathematical literacy.

/pdf/recognition-and-retrieval-of-mathematical-expressions-2x3ms5stvn.pdf

Recognition and retrieval of mathematical expressions

Recognition of on-line handwritten mathematical expressions using 2D stochastic context-free grammars and hidden Markov models

Scholarly digital libraries increasingly provide analytics to information within documents themselves. This includes information about the logical document structure of use to downstream components, such as search, navigation, and summarization. In this paper, the authors describe SectLabel, a module that further develops existing software to detect the logical structure of a document from existing PDF files, using the formalism of conditional random fields. While previous work has assumed access only to the raw text representation of the document, a key aspect of this work is to integrate the use of a richer representation of the document that includes features from optical character recognition OCR, such as font size and text position. Experiments reveal that using such rich features improves logical structure detection by a significant 9 F1 points, over a suitable baseline, motivating the use of richer document representations in other digital library applications.

/pdf/logical-structure-recovery-in-scholarly-articles-with-rich-34d7mea86b.pdf

Logical Structure Recovery in Scholarly Articles with Rich Document Features

A global learning approach for an online handwritten mathematical expression recognition system

Multimodal Integration.- Cross-Modality Matching of Loudness and Perceived Intensity of Whole-Body Vibrations.- Leaping across Modalities: Speed Regulation Messages in Audio and Tactile Domains.- The Effect of Spatial Disparity on the Integration of Auditory and Tactile Information.- Parametric Study of Virtual Curvature Recognition: Discrimination Thresholds for Haptic and Visual Sensory Information.- Cross-Modal Frequency Matching: Sound and Whole-Body Vibration.- Tactile and Sonic Explorations.- Audioworld: A Spatial Audio Tool for Acoustic and Cognitive Learning.- Exploring Interactive Systems Using Peripheral Sounds.- Basic Exploration of Narration and Performativity for Sounding Interactive Commodities.- Tactile Web Browsing for Blind Users.- Reducing Reversal Errors in Localizing the Source of Sound in Virtual Environment without Head Tracking.- Walking and Navigation Interfaces.- Conflicting Audio-haptic Feedback in Physically Based Simulation of Walking Sounds.- The Influence of Angle Size in Navigation Applications Using Pointing Gestures.- Audio-tactile Display of Ground Properties Using Interactive Shoes.- Efficient Acquisition of Force Data in Interactive Shoe Designs.- A Comparison of Two Wearable Tactile Interfaces with a Complementary Display in Two Orientations.- Prototype Design and Evaluation.- Virtual Sequencing with a Tactile Feedback Device.- The LapSlapper - Feel the Beat.- Product Design Review Application Based on a Vision-Sound-Haptic Interface.- The Phantom versus the Falcon: Force Feedback Magnitude Effects on User's Performance during Target Acquisition.- Gestures and Emotions.- Building a Framework for Communication of Emotional State through Interaction with Haptic Devices.- A Trajectory-Based Approach for Device Independent Gesture Recognition in Multimodal User Interfaces.

Haptic and audio interaction design : 5th international workshop, HAID 2010, Copenhagen, Denmark, September 16-17, 2010 : proceedings

In this paper, a statistical decision method for automatic classification of spatial relationships between each adjacent pair is proposed. Each pair is composed of mathematical symbols and/or alphabetical characters. Special treatment of mathematical symbols with variable size is important.This classification is important to recognize an accurate structure analysis module of math OCR. Experimental results on a very large database showed that the proposed method worked well with an accuracy of 99.57% by two important geometric feature relative size and relative position.

/pdf/statistical-classification-of-spatial-relationships-among-4to6wiu0fn.pdf

Statistical Classification of Spatial Relationships among Mathematical Symbols

This paper proposes the use of a formal grammar for the verification of mathematical formulae for a practical mathematical OCR system. Like a C compiler detecting syntax errors in a source file, we want to have a verification mechanism to find errors in the output of mathematical OCR. Linear monadic context-free tree grammar (LM-CFTG) was employed as a formal framework to define "well-formed" mathematical formulae. For the purpose of practical evaluation, a verification system for mathematical OCR was developed, and the effectiveness of the system was demonstrated by using the ground-truthed mathematical document database INFTY CDB-1.

/pdf/verification-of-mathematical-formulae-based-on-a-combination-2tkj6y6k3o.pdf

Verification of Mathematical Formulae Based on a Combination of Context-Free Grammar and Tree Grammar

We introduce a new universal design tactile graphics production system: BPLOT2. With BPLOT2, blind persons and sighted persons can share resources and cooperatively draw tactile graphics to be utilized in braille textbooks and teaching materials. Present tactile graphics design applications have only GUI (Graphical User Interface) with the need for mouse operations and thus are not usable by blind persons. Equally inconvenient, our self-supporting tactile graphics production system for the blind, BPLOT, has only CUI (Character User Interface), so BPLOT is not popular among the sighted. Therefore, we implemented GUI on BPLOT and realized universal design on BPLOT2.

/pdf/the-development-of-a-universal-design-tactile-graphics-4osarzqjbb.pdf

The Development of a Universal Design Tactile Graphics Production System BPLOT2

Synchronized CUI and GUI are developed for the universal design tactile graphics production system BPLOT3. BPLOT is the first tactile graphics production system for the blind that enables the blind to produce tactile graphics by themselves. With the new synchronized CUI and GUI of BPLOT3, the blind and the sighted can collaboratively produce tactile graphics. Proofreading of tactile graphics by a blind person is necessary in order to produce elaborate tactile graphics which can be used in textbooks or questions of entrance examinations. Because a blind person can modify tactile graphics by himself with BPLOT3, it will be a powerful tool.

/pdf/development-of-synchronized-cui-and-gui-for-universal-design-13z9fiiesk.pdf

Development of Synchronized CUI and GUI for Universal Design Tactile Graphics Production System BPLOT3

This study focuses on the class of string languages generated by TAGs It examines whether the class of string languages can be generated by some epsilon-free grammars and by some lexicalized grammars Utilizing spine grammars, this problem is solved positively This result for spine grammars can be translated into the result for TAGs

Akio Fujiyoshi

Papers

Statistical Classification of Spatial Relationships among Mathematical Symbols

Verification of Mathematical Formulae Based on a Combination of Context-Free Grammar and Tree Grammar

The Development of a Universal Design Tactile Graphics Production System BPLOT2

Development of Synchronized CUI and GUI for Universal Design Tactile Graphics Production System BPLOT3

Epsilon-Free Grammars and Lexicalized Grammars that Generate the Class of the Mildly Context-Sensitive Languages