scispace - formally typeset
Search or ask a question
Book ChapterDOI

Info-Graphics Retrieval: A Multi-kernel Distance Based Hashing Scheme

19 Dec 2016-pp 288-298
TL;DR: This paper presents a multi-modal document image retrieval framework by learning an optimal fusion of information from text and info-graphics regions and demonstrates the evaluation of the proposed concept on documents collected from various sources.
Abstract: Information retrieval research has shown significant improvement and provided techniques that retrieve documents in image or text form. However, retrieval of multi-modal documents has been given very less attention. We aim to build a system for retrieval of documents with embedded information graphics (Info-graphics). Info-graphics are images of bar charts and line graphs appearing with textual components in magazines, newspapers, and journals. In this paper, we present multi-modal document image retrieval framework by learning an optimal fusion of information from text and info-graphics regions. The evaluation of the proposed concept is demonstrated on documents collected from various sources such as magazines and journals.
References
More filters
Journal ArticleDOI
TL;DR: This work identifies the communicative signals that appear in simple bar charts, and presents an implemented Bayesian network methodology for reasoning about these signals and hypothesizing a bar chart's intended message.

53 citations

Proceedings ArticleDOI
12 Jun 2008
TL;DR: This paper outlines the approach to augmenting the core message of the graphic to produce a brief summary of a simple bar chart and simultaneously constructs both the discourse and sentence structures of the textual summary using a bottom-up approach.
Abstract: Information graphics, such as bar charts and line graphs, play an important role in multimodal documents. This paper presents a novel approach to producing a brief textual summary of a simple bar chart. It outlines our approach to augmenting the core message of the graphic to produce a brief summary. Our method simultaneously constructs both the discourse and sentence structures of the textual summary using a bottom-up approach. The result is then realized in natural language. An evaluation study validates our generation methodology.

37 citations

Posted Content
TL;DR: In this paper, a method for measuring the style similarity between infographics is presented, based on human perception data collected from crowdsourced experiments, using computer vision and machine learning algorithms to learn a style similarity metric for infographic designs.
Abstract: Infographics are complex graphic designs integrating text, images, charts and sketches. Despite the increasing popularity of infographics and the rapid growth of online design portfolios, little research investigates how we can take advantage of these design resources. In this paper we present a method for measuring the style similarity between infographics. Based on human perception data collected from crowdsourced experiments, we use computer vision and machine learning algorithms to learn a style similarity metric for infographic designs. We evaluate different visual features and learning algorithms and find that a combination of color histograms and Histograms-of-Gradients (HoG) features is most effective in characterizing the style of infographics. We demonstrate our similarity metric on a preliminary image retrieval test.

28 citations

Journal ArticleDOI
TL;DR: The scheme presents the extension of distance based hashing to kernel space for generating the indexing structure based on similarity in kernel space using the concept of multiple kernel learning to incorporate multiple features for defining the image indexing space.
Abstract: The paper presents a novel feature based indexing scheme for image collections. The scheme presents the extension of distance based hashing to kernel space for generating the indexing structure based on similarity in kernel space. The objective of the scheme is to incorporate multiple features for defining the image indexing space using the concept of multiple kernel learning. However, the indexing problems are defined with unique learning objective; therefore, a novel application of genetic algorithm is presented for the optimization task. The extensive evaluation of the proposed concept is performed for developing word based document indexing application of Devanagari, Bengali, and English scripts. In addition, the efficacy of the proposed concept is shown by experimental evaluations on handwritten digits and natural image collection.

22 citations

Book ChapterDOI
18 Jun 2014
TL;DR: A new methodology is presented that hypothesizes information needs from user queries and retrieves infographics based on how well the inherent structure and intended message of the graphics satisfy the query information needs.
Abstract: Information graphics, such as bar charts and line graphs, are a rich knowledge source that should be accessible to users. However, techniques that have been effective for document or image retrieval are inadequate for the retrieval of such graphics. We present and evaluate a new methodology that hypothesizes information needs from user queries and retrieves infographics based on how well the inherent structure and intended message of the graphics satisfy the query information needs.

14 citations