The subject about document image understanding is to extract and classify individual data meaningfully from paper-based documents. Until today, many methods/approaches have been proposed with regard to recognition of various kinds of documents, various technical problems for extensions of OCR, and requirements for practical usages. Of course, though the technical research issues in the early stage are looked upon as complementary attacks for the traditional OCR which is dependent on character recognition techniques, the application ranges or related issues are widely investigated or should be established progressively. This paper addresses current topics about document image understanding from a technical point of view as a survey. key words: document model, top-down, bottom-up, layout structure, logical structure, document types, layout recognition

Document Analysis and Recognition

Call for papers for Special Issue of ACM Transactions on Multimedia Computing, Communications and Applications on Interactive Digital Television

ACM Transactions on Multimedia Computing, Communications and Applications (ACM TOMCCAP)

We describe a method for automatically obtaining object representations suitable for retrieval from generic video shots. The object representation consists of an association of frame regions. These regions provide exemplars of the object's possible visual appearances.

Two ideas are developed: (i) associating regions within a single shot to represent a deforming object; (ii) associating regions from the multiple visual aspects of a 3D object, thereby implicitly representing 3D structure. For the association we exploit temporal continuity (tracking) and wide baseline matching of affine covariant regions.

In the implementation there are three areas of novelty: First, we describe a method to repair short gaps in tracks. Second, we show how to join tracks across occlusions (where many tracks terminate simultaneously). Third, we develop an affine factorization method that copes with motion degeneracy.

We obtain tracks that last throughout the shot, without requiring a 3D reconstruction. The factorization method is used to associate tracks into object-level groups, with common motion. The outcome is that separate parts of an object that are not simultaneously visible (such as the front and back of a car, or the front and side of a face) are associated together. In turn this enables object-level matching and recognition throughout a video.

We illustrate the method on the feature film "Groundhog Day." Examples are given for the retrieval of deforming objects (heads, walking people) and rigid objects (vehicles, locations).

/pdf/object-level-grouping-for-video-shots-1vsfqifb5l.pdf

Object Level Grouping for Video Shots

In India, more than 300 million people use Devanagari script for documentation. There has been a significant improvement in the research related to the recognition of printed as well as handwritten Devanagari text in the past few years. State of the art from 1970s of machine printed and handwritten Devanagari optical character recognition (OCR) is discussed in this paper. All feature-extraction techniques as well as training, classification and matching techniques useful for the recognition are discussed in various sections of the paper. An attempt is made to address the most important results reported so far and it is also tried to highlight the beneficial directions of the research till date. Moreover, the paper also contains a comprehensive bibliography of many selected papers appeared in reputed journals and conference proceedings as an aid for the researchers working in the field of Devanagari OCR.

Offline Recognition of Devanagari Script: A Survey

Table detection is a crucial step in many document analysis applications as tables are used for presenting essential information to the reader in a structured manner. It is a hard problem due to varying layouts and encodings of the tables. Researchers have proposed numerous techniques for table detection based on layout analysis of documents. Most of these techniques fail to generalize because they rely on hand engineered features which are not robust to layout variations. In this paper, we have presented a deep learning based method for table detection. In the proposed method, document images are first pre-processed. These images are then fed to a Region Proposal Network followed by a fully connected neural network for table detection. The proposed method works with high precision on document images with varying layouts that include documents, research papers, and magazines. We have done our evaluations on publicly available UNLV dataset where it beats Tesseract's state of the art table detection system by a significant margin.

Table Detection Using Deep Learning

There are many exercises which are repetitive in nature and are required to be done with perfection to derive maximum benefits. Sun Salutation or Surya Namaskar is one of the oldest yoga practice known. It is a sequence of ten actions or 'asanas' where the actions are synchronized with breathing and each action and its transition should be performed with minimal jerks. Essentially, it is important that this yoga practice be performed with Grace and Consistency. In this context, Grace is the ability of a person to perform an exercise with smoothness i.e. without sudden movements or jerks during the posture transition and Consistency measures the repeatability of an exercise in every cycle. We propose an algorithm that assesses how well a person practices Sun Salutation in terms of grace and consistency. Our approach works by training individual HMMs for each asana using STIP features[11] followed by automatic segmentation and labeling of the entire Sun Salutation sequence using a concatenated-HMM. The metric of grace and consistency are then laid down in terms of posture transition times. The assessments made by our system are compared with the assessments of the yoga trainer to derive the accuracy of the system. We introduce a dataset for Sun Salutation videos comprising 30 sequences of perfect Sun Salutation performed by seven experts and used this dataset to train our system. While Sun Salutation can be judged on multiple parameters, we focus mainly on judging Grace and Consistency.

A framework to assess Sun salutation videos

The thinning methodology is novel in terms of its ability to incorporate character shape specific knowledge while constructing the thinned skeleton. But removal of spurious strokes or shape deformation in thinning is a difficult problem. In this paper, we have proposed a novel medial-axis based thinning strategy used for performing skeletonization of noisy character images. The proposed algorithm produces segmented strokes in vector form as a by-product. Hence further stroke segmentation is not required. Experiment is done on printed English, Bengali, Hindi, and Tamil characters and we obtain less spurious branches compared to other thinning methods without any post processing. We have concluded with a proposed methodology to extract structural features from thinned character images. This feature set improves the performance of existing OCR for Indian languages.

A medial axis based thinning strategy and structural feature extraction of character images

A novel technique for binarization of degraded documents is proposed. It works in a multi-scale framework with an adaptive-cum-interpolative thresholding as a modification of Otsu's method. Instead of computing a global threshold value for an input document image, it computes the local threshold values for a small set of grid points by observing the intensity pattern of the pixels lying in the concerned grid cells. Thresholds estimated for these grid points are used, in turn, to compute the threshold values of all the remaining pixels using a fast-yet-efficient interpolation procedure. To handle noises in degraded images, this grid-based adaptive thresholding is applied in successively reducing scales to obtain the nearoptimal binarization as a set of connected components. After a post-processing with these connected components, we get the final output. Exhaustive experimentation has been carried out with benchmark datasets including George Washington corpus of handwritten documents, and also with our own datasets. When compared to other methods, the proposed method is found to be robust and appreciably better, as tested by conventional evaluation schemes.

Robust binarization of degraded documents using adaptive-cum-interpolative thresholding in a multi-scale framework

We present Heritage+, an integrated platform for interactive access of different types of media elements through an unified interface. A unique aspect of Heritage+ is that it deals with document images as distinct media type and implements tools and techniques for browsing and querying document images along with other media elements like video sequences and images. Further, Heritage+ proposes a new scheme for encoding and use of ontology for accessing multimedia collection. In the context of document images, the ontology specifies the document class-specific semantics of the logical components that help in an automated semantically meaningful linking of documents and their components with heterogeneous media-type resources. Further, Heritage+ supports conceptual query of document images along with other media elements. This multifunctional access interface to the document images is provided in Heritage+ using a novel model guided document image segmentation scheme and word-image based indexing scheme.

Managing document images in a digital library: an ontology guided approach

In this paper, we present novel topological features based on the structural shape of a character. We detect the convexshaped segments formed by the various strokes. The convex segments are then represented with shape primitives from a repertoire. The character is represented as a spatial layout of convex segments. We formulate feature templates for Bangla characters. A given character is assigned the label of the best matching feature template. We have tested the method on a benchmark datasets of printed and handwritten Bangla basic and compound character images. Our results demonstrate the efficacy of our approach.

Gaurav Harit

Papers

A framework to assess Sun salutation videos

A medial axis based thinning strategy and structural feature extraction of character images

Robust binarization of degraded documents using adaptive-cum-interpolative thresholding in a multi-scale framework

Managing document images in a digital library: an ontology guided approach

Topological features for recognizing printed and handwritten Bangla characters