Hiranmay Ghosh

Bio: Hiranmay Ghosh is an academic researcher from Tata Consultancy Services. The author has contributed to research in topics: Ontology (information science) & Multimedia Web Ontology Language. The author has an hindex of 12, co-authored 48 publications receiving 415 citations. Previous affiliations of Hiranmay Ghosh include Indian Institutes of Technology & Harvard University.

Nrityakosha: Preserving the intangible heritage of Indian classical dance

Abstract: Preservation of intangible cultural heritage, such as music and dance, requires encoding of background knowledge together with digitized records of the performances. We present an ontology-based approach for designing a cultural heritage repository for that purpose. Since dance and music are recorded in multimedia format, we use Multimedia Web Ontology Language (MOWL) to encode the domain knowledge. We propose an architectural framework that includes a method to construct the ontology with a labeled set of training data and use of the ontology to automatically annotate new instances of digital heritage artifacts. The annotations enable creation of a semantic navigation environment in a cultural heritage repository. We have demonstrated the efficacy of our approach by constructing an ontology for the cultural heritage domain of Indian classical dance, and have developed a browsing application for semantic access to the heritage collection of Indian dance videos.

Ontology Specification and Integration for Multimedia Applications

Abstract: An ontology designed for multimedia applications should enable integration of the conceptual and media spaces. We present M-OWL, a new ontology language, that supports this capability. M-OWL supports explicit definition of media properties for the concepts. The language has been defined as an extension of OWL, the standard ontology language for the web. We have proposed a new Bayesian Network based probabilistic reasoning framework with M-OWL for semantic interpretation of multimedia data. We have also proposed a new model for ontology integration, based on the similarity of the concepts in the media domain. It can be used to integrate several multimedia and traditional ontologies.

Telecom Inventory Management via Object Recognition and Localisation on Google Street View Images

Abstract: We present a novel method to update assets for telecommunication infrastructure using google street view (GSV) images. The problem is formulated as a object recognition task, followed by use of triangulation to estimate the object coordinates from sensor plane coordinates, To this end, we have explored different state-of-the-art object recognition techniques both from feature engineering and using deep learning namely HOG descriptors with SVM, Deformable parts model (DPM), and Deep learning (DL) using faster RCNNs. While HOG+SVM has proved to be robust human detector, DPM which is based on probabilistic graphical models and DL which is a non-linear classifier have proved their versatility in different types of object recognition problems. Asset recognition from the street view images however pose unique challenge as they could be installed on the ground in various poses, orientations and with occlusions, objects camouflaged in the background and in some cases inter class variation is small. We present comparative performance of these techniques for specific use-case involving telecom equipment for highest precision and recall. The blocks of proposed pipeline are detailed and compared to traditional inventory management methods.

MOWL: An ontology representation language for web-based multimedia applications

Abstract: Several multimedia applications need to reason with concepts and their media properties in specific domain contexts. Media properties of concepts exhibit some unique characteristics that cannot be dealt with conceptual modeling schemes followed in the existing ontology representation and reasoning schemes. We have proposed a new perceptual modeling technique for reasoning with media properties observed in multimedia instances and the latent concepts. Our knowledge representation scheme uses a causal model of the world where concepts manifest in media properties with uncertainties. We introduce a probabilistic reasoning scheme for belief propagation across domain concepts through observation of media properties. In order to support the perceptual modeling and reasoning paradigm, we propose a new ontology language, Multimedia Web Ontology Language (MOWL). Our primary contribution in this article is to establish the need for the new ontology language and to introduce the semantics of its novel language constructs. We establish the generality of our approach with two disperate knowledge-intensive applications involving reasoning with media properties of concepts.

An Ontology Based Personalized Garment Recommendation System

Abstract: We present a novel method for content-based recommendation of media-rich commodities using probabilistic multimedia ontology. The ontology encodes subjective knowledge of experts that enables interpretation of media based and semantic product features in context of domain concepts. Our recommendation is based on semantic compatibility between the products and user profile in context of use. We use probabilistic knowledge representation and reasoning framework to achieve robust and flexible results. The approach has been validated with fashion preferences of several individuals with a large collection of Sarees, an ethnic dress for women in Indian subcontinent.

Pattern Recognition and Machine Learning

Abstract: Probability Distributions.- Linear Models for Regression.- Linear Models for Classification.- Neural Networks.- Kernel Methods.- Sparse Kernel Machines.- Graphical Models.- Mixture Models and EM.- Approximate Inference.- Sampling Methods.- Continuous Latent Variables.- Sequential Data.- Combining Models.

A Survey on Visual Content-Based Video Indexing and Retrieval

Abstract: Video indexing and retrieval have a wide spectrum of promising applications, motivating the interest of researchers worldwide. This paper offers a tutorial and an overview of the landscape of general strategies in visual content-based video indexing and retrieval, focusing on methods for video structure analysis, including shot boundary detection, key frame extraction and scene segmentation, extraction of features including static key frame features, object features and motion features, video data mining, video annotation, video retrieval including query interfaces, similarity measure and relevance feedback, and video browsing. Finally, we analyze future research directions.

Texture Feature Extraction Methods: A Survey

Abstract: Texture analysis is used in a very broad range of fields and applications, from texture classification (e.g., for remote sensing) to segmentation (e.g., in biomedical imaging), passing through image synthesis or pattern recognition (e.g., for image inpainting). For each of these image processing procedures, first, it is necessary to extract—from raw images—meaningful features that describe the texture properties. Various feature extraction methods have been proposed in the last decades. Each of them has its advantages and limitations: performances of some of them are not modified by translation, rotation, affine, and perspective transform; others have a low computational complexity; others, again, are easy to implement; and so on. This paper provides a comprehensive survey of the texture feature extraction methods. The latter are categorized into seven classes: statistical approaches, structural approaches, transform-based approaches, model-based approaches, graph-based approaches, learning-based approaches, and entropy-based approaches. For each method in these seven classes, we present the concept, the advantages, and the drawbacks and give examples of application. This survey allows us to identify two classes of methods that, particularly, deserve attention in the future, as their performances seem interesting, but their thorough study is not performed yet.

ACM Transactions on Multimedia Computing, Communications and Applications (ACM TOMCCAP)

Abstract: Call for papers for Special Issue of ACM Transactions on Multimedia Computing, Communications and Applications on Interactive Digital Television