The subject about document image understanding is to extract and classify individual data meaningfully from paper-based documents. Until today, many methods/approaches have been proposed with regard to recognition of various kinds of documents, various technical problems for extensions of OCR, and requirements for practical usages. Of course, though the technical research issues in the early stage are looked upon as complementary attacks for the traditional OCR which is dependent on character recognition techniques, the application ranges or related issues are widely investigated or should be established progressively. This paper addresses current topics about document image understanding from a technical point of view as a survey. key words: document model, top-down, bottom-up, layout structure, logical structure, document types, layout recognition

Document Analysis and Recognition

Call for papers for Special Issue of ACM Transactions on Multimedia Computing, Communications and Applications on Interactive Digital Television

ACM Transactions on Multimedia Computing, Communications and Applications (ACM TOMCCAP)

We describe a method for automatically obtaining object representations suitable for retrieval from generic video shots. The object representation consists of an association of frame regions. These regions provide exemplars of the object's possible visual appearances.

Two ideas are developed: (i) associating regions within a single shot to represent a deforming object; (ii) associating regions from the multiple visual aspects of a 3D object, thereby implicitly representing 3D structure. For the association we exploit temporal continuity (tracking) and wide baseline matching of affine covariant regions.

In the implementation there are three areas of novelty: First, we describe a method to repair short gaps in tracks. Second, we show how to join tracks across occlusions (where many tracks terminate simultaneously). Third, we develop an affine factorization method that copes with motion degeneracy.

We obtain tracks that last throughout the shot, without requiring a 3D reconstruction. The factorization method is used to associate tracks into object-level groups, with common motion. The outcome is that separate parts of an object that are not simultaneously visible (such as the front and back of a car, or the front and side of a face) are associated together. In turn this enables object-level matching and recognition throughout a video.

We illustrate the method on the feature film "Groundhog Day." Examples are given for the retrieval of deforming objects (heads, walking people) and rigid objects (vehicles, locations).

/pdf/object-level-grouping-for-video-shots-1vsfqifb5l.pdf

Object Level Grouping for Video Shots

In India, more than 300 million people use Devanagari script for documentation. There has been a significant improvement in the research related to the recognition of printed as well as handwritten Devanagari text in the past few years. State of the art from 1970s of machine printed and handwritten Devanagari optical character recognition (OCR) is discussed in this paper. All feature-extraction techniques as well as training, classification and matching techniques useful for the recognition are discussed in various sections of the paper. An attempt is made to address the most important results reported so far and it is also tried to highlight the beneficial directions of the research till date. Moreover, the paper also contains a comprehensive bibliography of many selected papers appeared in reputed journals and conference proceedings as an aid for the researchers working in the field of Devanagari OCR.

Offline Recognition of Devanagari Script: A Survey

Table detection is a crucial step in many document analysis applications as tables are used for presenting essential information to the reader in a structured manner. It is a hard problem due to varying layouts and encodings of the tables. Researchers have proposed numerous techniques for table detection based on layout analysis of documents. Most of these techniques fail to generalize because they rely on hand engineered features which are not robust to layout variations. In this paper, we have presented a deep learning based method for table detection. In the proposed method, document images are first pre-processed. These images are then fed to a Region Proposal Network followed by a fully connected neural network for table detection. The proposed method works with high precision on document images with varying layouts that include documents, research papers, and magazines. We have done our evaluations on publicly available UNLV dataset where it beats Tesseract's state of the art table detection system by a significant margin.

Table Detection Using Deep Learning

This paper presents a new approach to build distributed digital libraries with multimedia contents. The authors propose a new scheme for media feature based concept modelling to address the limitation of traditional ontology based multimedia retrieval systems. The perceptual models can be used for semantic query processing using standard MPEG-7 media content descriptions. The authors have defined a new ontology language M-OWL (multimedia web ontology language) to support this perceptual modelling. M-OWL is an extension to the OWL (web ontology language) with new constructs for formal representation of the media properties of the domain concepts. It supports probabilistic evidential reasoning for robust concept recognition in multimedia documents. The separation of perceptual modelling of concepts from the repository architecture enables seamless integration of diverse multimedia contents. SOA (Service Oriented Architecture) is used to integrate large number of distributed information sources, each of which is modelled as an intelligent information agent. The authors have demonstrated the capability of the architecture by building a few research prototypes, namely a virtual encyclopaedia of Indian culture, a document image repository and a multimedia portal.

Using ontology for building distributed digital libraries with multimedia contents

Structural analysis helps in parsing the mathematical expressions. Various approaches for structural analysis have been reported in literature, but they mainly deal with online and printed expressions. In this work, two-dimensional, stochastic context-free grammar is used for the structural analysis of offline handwritten mathematical expressions in a document image. The spatial relation between characters in an expression has been incorporated so that the structural variability in handwritten expressions can be tackled.

Structural Analysis of Offline Handwritten Mathematical Expressions

Preprocessing techniques form an important task in document image analysis. In structured documents like forms, cheques, etc., there is a predefined space called frame field/cell for the user to fill the entry. When the user is writing, the nonuniformity of inter-character spacing becomes an issue. Many times, the starting characters of the word are written with sparse spacing between the characters and then gradually with a more compact spacing so as to accommodate the word within the frame field. To deal with this variation in intra-word spacing, horizontal-scale correction is applied to the extracted form fields. The effectiveness of the system is proved by applying it as a preprocessing step in a recognition system proposed in (Almazan et al. in Pattern Anal Mach Intell 36(12):21552–2566, 2014 [2]). The recognition framework results in reduced error rates with this normalization.

Cell Extraction and Horizontal-Scale Correction in Structured Documents

E-book refers to books in Electronic form. In this paper we provide a review of the technological issues involved in design and construction of e-books. We also address the issues involved in designing e-books in Indian languages. We have proposed a novel scheme for construction and delivery of legacy documents (in Indian and other languages) in the form of e-books with the facility of hyper-linking and indexing of various logical components in the document image.

A New Look at Books: E-books

This paper presents an Optical Character Recognition (OCR) system for documents with English text and mathematical expressions. Neural network architectures using CNN layers and/or dense layers achieve high level accuracy in character recognition task. However, these models require large amount of data to train the network, with balanced number of samples for each class. Recognition of mathematical symbols poses challenges of the imbalance and paucity of training data available. To address this issue, we pose the character recognition problem as a Distance Metric Learning problem. We propose a Siamese-CNN Network that learns discriminative features to identify if the two images in a pair contain similar or dissimilar characters. The network is then used to recognize different characters by character matching where test images are compared to sample images of any target class which may or may not be included during training. Thus our model can scale to new symbols easily. The proposed approach is invariant to author’s handwriting. Our model has been tested over images extracted from a dataset of scanned answer scripts collected by us. It is seen that our approach achieves comparable performance to other architectures using convolutional layers or dense layers while using lesser training data.

Gaurav Harit

Papers

Using ontology for building distributed digital libraries with multimedia contents

Structural Analysis of Offline Handwritten Mathematical Expressions

Cell Extraction and Horizontal-Scale Correction in Structured Documents

A New Look at Books: E-books

DocDescribor: Digits + Alphabets + Math Symbols - A Complete OCR for Handwritten Documents