The subject about document image understanding is to extract and classify individual data meaningfully from paper-based documents. Until today, many methods/approaches have been proposed with regard to recognition of various kinds of documents, various technical problems for extensions of OCR, and requirements for practical usages. Of course, though the technical research issues in the early stage are looked upon as complementary attacks for the traditional OCR which is dependent on character recognition techniques, the application ranges or related issues are widely investigated or should be established progressively. This paper addresses current topics about document image understanding from a technical point of view as a survey. key words: document model, top-down, bottom-up, layout structure, logical structure, document types, layout recognition

Document Analysis and Recognition

Call for papers for Special Issue of ACM Transactions on Multimedia Computing, Communications and Applications on Interactive Digital Television

ACM Transactions on Multimedia Computing, Communications and Applications (ACM TOMCCAP)

We describe a method for automatically obtaining object representations suitable for retrieval from generic video shots. The object representation consists of an association of frame regions. These regions provide exemplars of the object's possible visual appearances.

Two ideas are developed: (i) associating regions within a single shot to represent a deforming object; (ii) associating regions from the multiple visual aspects of a 3D object, thereby implicitly representing 3D structure. For the association we exploit temporal continuity (tracking) and wide baseline matching of affine covariant regions.

In the implementation there are three areas of novelty: First, we describe a method to repair short gaps in tracks. Second, we show how to join tracks across occlusions (where many tracks terminate simultaneously). Third, we develop an affine factorization method that copes with motion degeneracy.

We obtain tracks that last throughout the shot, without requiring a 3D reconstruction. The factorization method is used to associate tracks into object-level groups, with common motion. The outcome is that separate parts of an object that are not simultaneously visible (such as the front and back of a car, or the front and side of a face) are associated together. In turn this enables object-level matching and recognition throughout a video.

We illustrate the method on the feature film "Groundhog Day." Examples are given for the retrieval of deforming objects (heads, walking people) and rigid objects (vehicles, locations).

/pdf/object-level-grouping-for-video-shots-1vsfqifb5l.pdf

Object Level Grouping for Video Shots

In India, more than 300 million people use Devanagari script for documentation. There has been a significant improvement in the research related to the recognition of printed as well as handwritten Devanagari text in the past few years. State of the art from 1970s of machine printed and handwritten Devanagari optical character recognition (OCR) is discussed in this paper. All feature-extraction techniques as well as training, classification and matching techniques useful for the recognition are discussed in various sections of the paper. An attempt is made to address the most important results reported so far and it is also tried to highlight the beneficial directions of the research till date. Moreover, the paper also contains a comprehensive bibliography of many selected papers appeared in reputed journals and conference proceedings as an aid for the researchers working in the field of Devanagari OCR.

Offline Recognition of Devanagari Script: A Survey

Table detection is a crucial step in many document analysis applications as tables are used for presenting essential information to the reader in a structured manner. It is a hard problem due to varying layouts and encodings of the tables. Researchers have proposed numerous techniques for table detection based on layout analysis of documents. Most of these techniques fail to generalize because they rely on hand engineered features which are not robust to layout variations. In this paper, we have presented a deep learning based method for table detection. In the proposed method, document images are first pre-processed. These images are then fed to a Region Proposal Network followed by a fully connected neural network for table detection. The proposed method works with high precision on document images with varying layouts that include documents, research papers, and magazines. We have done our evaluations on publicly available UNLV dataset where it beats Tesseract's state of the art table detection system by a significant margin.

Table Detection Using Deep Learning

We target the problem of Image Denoising using Gaussian Processes Regression (GPR). Being a non-parametric regression technique, GPR has received much attention in the recent past and here we further explore its versatility by applying it to a denoising problem. The focus is primarily on the design of a local gradient sensitive kernel that captures pixel similarity in the context of image denoising. This novel kernel formulation is used to shape the smoothness of the joint GP prior. We apply the GPR denoising technique to small patches and then stitch back these patches, this allows the priors to be local and relevant, also this helps us in dealing with GPR complexity. We demonstrate that our GPR based technique gives better PSNR values in comparison to existing popular denoising techniques.

Gradient sensitive kernel for Image Denoising, using Gaussian Process Regression

In this paper, we improve the performance of the recently proposed Direct Query Classifier (dqc). The (dqc) is a classifier based retrieval method and in general, such methods have been shown to be superior to the OCR-based solutions for performing retrieval in many practical document image datasets. In (dqc), the classifiers are trained for a set of frequent queries and seamlessly extended for the rare and arbitrary queries. This extends the classifier based retrieval paradigm to an unlimited number of classes (words) present in a language. The (dqc) requires indexing cut-portions (n-grams) of the word image and dtw distance has been used for indexing. However, dtw is computationally slow and therefore limits the performance of the (dqc). We introduce query specific dtw distance, which enables effective computation of global principal alignments for novel queries. Since the proposed query specific dtw distance is a linear approximation of the dtw distance, it enhances the performance of the (dqc). Unlike previous approaches, the proposed query specific dtw distance uses both the class mean vectors and the query information for computing the global principal alignments for the query. Since the proposed method computes the global principal alignments using n-grams, it works well for both frequent and rare queries. We also use query expansion (qe) to further improve the performance of our query specific dtw. This also allows us to seamlessly adapt our solution to new fonts, styles and collections. We have demonstrated the utility of the proposed technique over 3 different datasets. The proposed query specific dtw performs well compared to the previous dtw approximations.

https://www.mdpi.com/2313-433X/4/2/37/pdf

Efficient Query Specific DTW Distance for Document Retrieval with Unlimited Vocabulary

We propose an empirical computational model for generating an interpretation of a video shot based on our proposed principle of perceptual prominence. The principle of perceptual prominence captures the key aspects of mise-en-scene required for interpreting a video scene. We present a novel approach for applying perceptual grouping principles to the spatio-temporal domain of video. Our spatio-temporal perceptual grouping scheme, applied on blob tracks, makes use of a specified spatio-temporal coherence model. A high level semantic interpretation of scenes is done using the mise-en-scene features and the perceptual prominence computed for the perceptual clusters.

Video Scene Interpretation Using Perceptual Prominence and Mise-en-scène Features

In this paper, we present novel features based on the topography of a character as visible from different viewing directions on a 2D plane. By topography of a character we mean the structural features of the strokes and their spatial relations. In this work we develop topographic features of strokes visible with respect to views from different directions (e.g. North, South, East, and West). We consider three types of topographic features: closed region, convexity of strokes, and straight line strokes. We have tested the proposed method on printed and handwritten Bengali and Hindi isolated character images. Initial results demonstrate the efficacy of our approach.

A Novel Topographic Feature Extraction Method for Indian Character Images

This paper addresses the problem of segmenting handwritten annotations on scientific research papers. The motivation of this work is to geometrically segment the complex cases of handwritten annotations, including marks, cuts and special symbols along, with the regular text. Our work particularly focuses on documents that have multi-oriented handwritten [1] annotations rather than annotations in controlled scenario [2]. Spectral Partitioning is adopted as the segmentation scheme to separate the printed text and annotations. A new feature Envelope Straightness is developed and included in our feature set. This leads to an improvement of accuracy over the state-of-the-art features. The experiments are performed on two datasets: 40 documents authored by two writers from IAM dataset, comprising only printed and handwritten text and a self created dataset of 40 scientific papers from various proceedings annotated by a reader, comprising varied types of annotations. In the framework of spectral partitioning, our feature set has achieved a recall of 98.39% for printed text and precision of 85.40% for handwritten annotations on our dataset. For IAM dataset our feature set has achieved a recall of 81.89% for printed text and a precision of 69.67% for handwritten annotations. The results achieved on both dataset are better compared with results obtained using [3] [1].

Gaurav Harit

Papers

Gradient sensitive kernel for Image Denoising, using Gaussian Process Regression

Efficient Query Specific DTW Distance for Document Retrieval with Unlimited Vocabulary

Video Scene Interpretation Using Perceptual Prominence and Mise-en-scène Features

A Novel Topographic Feature Extraction Method for Indian Character Images

Segmenting printed text and handwritten annotation by Spectral Partitioning