Learning ontology for personalized video retrieval
28 Sep 2007-pp 39-46
TL;DR: A reinforcement learning algorithm is proposed for the parameters of the Bayesian Network with the implicit feedback obtained from the clickthrough data to provide personalized ranking of results in a video retrieval system.
Abstract: This paper proposes a new method for using implicit user feedback from clickthrough data to provide personalized ranking of results in a video retrieval system. The annotation based search is complemented with a feature based ranking in our approach. The ranking algorithm uses belief revision in a Bayesian Network, which is derived from a multimedia ontology that captures the probabilistic association of a concept with expected video features. We have developed a content model for videos using discrete feature states to enable Bayesian reasoning and to alleviate on-line feature processing overheads. We propose a reinforcement learning algorithm for the parameters of the Bayesian Network with the implicit feedback obtained from the clickthrough data.
Citations
More filters
01 Nov 2011
TL;DR: Methods for video structure analysis, including shot boundary detection, key frame extraction and scene segmentation, extraction of features including static key frame features, object features and motion features, video data mining, video annotation, and video retrieval including query interfaces are analyzed.
Abstract: Video indexing and retrieval have a wide spectrum of promising applications, motivating the interest of researchers worldwide. This paper offers a tutorial and an overview of the landscape of general strategies in visual content-based video indexing and retrieval, focusing on methods for video structure analysis, including shot boundary detection, key frame extraction and scene segmentation, extraction of features including static key frame features, object features and motion features, video data mining, video annotation, video retrieval including query interfaces, similarity measure and relevance feedback, and video browsing. Finally, we analyze future research directions.
606 citations
TL;DR: This survey reviews the interesting features that can be extracted from video data for indexing and retrieval along with similarity measurement methods and identifies present research issues in area of content based video retrieval systems.
Abstract: With the development of multimedia data types and available bandwidth there is huge demand of video retrieval systems, as users shift from text based retrieval systems to content based retrieval systems. Selection of extracted features play an important role in content based video retrieval regardless of video attributes being under consideration. These features are intended for selecting, indexing and ranking according to their potential interest to the user. Good features selection also allows the time and space costs of the retrieval process to be reduced. This survey reviews the interesting features that can be extracted from video data for indexing and retrieval along with similarity measurement methods. We also identify present research issues in area of content based video retrieval systems.
90 citations
Posted Content•
TL;DR: A novel content-based heterogeneous information retrieval framework, particularly well suited to browse medical databases and support new generation computer aided diagnosis (CADx) systems, is presented in this paper.
Abstract: A novel content-based heterogeneous information retrieval framework, particularly well suited to browse medical databases and support new generation computer aided diagnosis (CADx) systems, is presented in this paper. It was designed to retrieve possibly incomplete documents, consisting of several images and semantic information, from a database; more complex data types such as videos can also be included in the framework.
52 citations
TL;DR: In this paper, a content-based heterogeneous information retrieval framework was proposed to retrieve possibly incomplete documents, consisting of several images and semantic information, from a database; more complex data types such as videos can also be included in the framework.
Abstract: A novel content-based heterogeneous information retrieval framework, particularly well suited to browse medical databases and support new generation computer aided diagnosis (CADx) systems, is presented in this paper. It was designed to retrieve possibly incomplete documents, consisting of several images and semantic information, from a database; more complex data types such as videos can also be included in the framework. The proposed retrieval method relies on image processing, in order to characterize each individual image in a document by their digital content, and information fusion. Once the available images in a query document are characterized, a degree of match, between the query document and each reference document stored in the database, is defined for each attribute (an image feature or a metadata). A Bayesian network is used to recover missing information if need be. Finally, two novel information fusion methods are proposed to combine these degrees of match, in order to rank the reference documents by decreasing relevance for the query. In the first method, the degrees of match are fused by the Bayesian network itself. In the second method, they are fused by the Dezert-Smarandache theory: the second approach lets us model our confidence in each source of information (i.e., each attribute) and take it into account in the fusion process for a better retrieval performance. The proposed methods were applied to two heterogeneous medical databases, a diabetic retinopathy database and a mammography screening database, for computer aided diagnosis. Precisions at five of 0.809 ± 0.158 and 0.821 ± 0.177, respectively, were obtained for these two databases, which is very promising.
50 citations
TL;DR: This paper forms the personalized video big data retrieval problem as an interaction between the user and the system via a stochastic process, not just a similarity matching, accuracy (feedback) model of the retrieval; introduces users’ real-time context into the retrieval system; and proposes a general framework.
Abstract: Online video sharing (e.g., via YouTube or YouKu) has emerged as one of the most important services in the current Internet, where billions of videos on the cloud are awaiting exploration. Hence, a personalized video retrieval system is needed to help users find interesting videos from big data content. Two of the main challenges are to process the increasing amount of video big data and resolve the accompanying “cold start” issue efficiently. Another challenge is to satisfy the users’ need for personalized retrieval results, of which the accuracy is unknown. In this paper, we formulate the personalized video big data retrieval problem as an interaction between the user and the system via a stochastic process, not just a similarity matching, accuracy (feedback) model of the retrieval; introduce users’ real-time context into the retrieval system; and propose a general framework for this problem. By using a novel contextual multiarmed bandit-based algorithm to balance the accuracy and efficiency, we propose a context-based online big-data-oriented personalized video retrieval system. This system can support datasets that are dynamically increasing in size and has the property of cross-modal retrieval. Our approach provides accurate retrieval results with sublinear regret and linear storage complexity and significantly improves the learning speed. Furthermore, by learning for a cluster of similar contexts simultaneously, we can realize sublinear storage complexity with the same regret but slightly poorer performance on the “cold start” issue compared to the previous approach. We validate our theoretical results experimentally on a tremendously large dataset; the results demonstrate that the proposed algorithms outperform existing bandit-based online learning methods in terms of accuracy and efficiency and the adaptation from the bandit framework offers additional benefits.
25 citations
Cites background from "Learning ontology for personalized ..."
...feedback to provide personalized retrieval output in [28], [29]....
[...]
References
More filters
Proceedings Article•
01 Aug 1997TL;DR: This paper introduces a new approach that allows for the flexible manipulation of the tradeoff between the quality of the learned networks and the amount of information that is maintained about past observations and evaluates its effectiveness through and empirical study.
Abstract: There is an obvious need for improving the performance and accuracy of a Bayesian network as new data is observed. Because of errors in model construction and changes in the dynamics of the domains, we cannot afford to ignore the information in new data. While sequential update of parameters for a fixed structure can be accomplished using standard techniques, sequential update of network structure is still an open problem. In this paper, we investigate sequential update of Bayesian networks were both parameters and structure are expected to change. We introduce a new approach that allows for the flexible manipulation of the tradeoff between the quality of the learned networks and the amount of information that is maintained about past observations. We formally describe our approach including the necessary modifications to the scoring functions for learning Bayesian networks, evaluate its effectiveness through and empirical study, and extend it to the case of missing data.
137 citations
25 Jul 2004
TL;DR: This paper extracts a set of predictive features from the thread trees of newsgroup messages as well as features of message authors and lexical distribution within a message thread to create an effective ranking function to predict the most relevant messages to queries in community search.
Abstract: Web communities are web virtual broadcasting spaces where people can freely discuss anything. While such communities function as discussion boards, they have even greater value as large repositories of archived information. In order to unlock the value of this resource, we need an effective means for searching archived discussion threads. Unfortunately the techniques that have proven successful for searching document collections and the Web are not ideally suited to the task of searching archived community discussions. In this paper, we explore the problem of creating an effective ranking function to predict the most relevant messages to queries in community search. We extract a set of predictive features from the thread trees of newsgroup messages as well as features of message authors and lexical distribution within a message thread. Our final results indicate that when using linear regression with this feature set, our search system achieved a 28.5% performance improvement compared to our baseline system.
67 citations
Additional excerpts
...…A retrieval system uses some functions to establish a similarity score between every document in a collection and a Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for…...
[...]
01 Jan 1999
TL;DR: The theory of IPF is investigated with a mathematical definition of the procedure and a review of the relevant literature given, and the procedure has been automated in Visual Basic and a description of the program and a ‘User Guide’ are provided.
Abstract: ‘Iterative Proportional Fitting’ (IPF) is a mathematical procedure originally developed to combine the information from two or more datasets. IPF is a well-established technique with the theoretical and practical considerations behind the method thoroughly explored and reported.
In this paper the theory of IPF is investigated with a mathematical definition of the procedure and a review of the relevant literature given. So that IPF can be readily accessible to researchers the procedure has been automated in Visual Basic and a description of the program and a ‘User Guide’ are provided.
IPF is employed in various disciplines but has been particularly useful in census-related analysis to provide updated population statistics and to estimate individual-level attribute characteristics. To illustrate the practical application of IPF various case studies are described. In the future, demand for individual-level data is thought likely to increase and it is believed that the IPF procedure and Visual Basic program have the potential to facilitate research in geography and other disciplines.
67 citations
06 Nov 2005
TL;DR: A novel framework where individual components are developed to model different relationships between documents and queries and then combined into a joint retrieval framework is proposed, which demonstrates over 14 % improvement in IR performance over the best reported text-only baseline and ranks amongst the best results reported on this corpus.
Abstract: In this paper we describe a novel approach for jointly modeling the text and the visual components of multimedia documents for the purpose of information retrieval(IR). We propose a novel framework where individual components are developed to model different relationships between documents and queries and then combined into a joint retrieval framework. In the state-of-the-art systems, a late combination between two independent systems, one analyzing just the text part of such documents, and the other analyzing the visual part without leveraging any knowledge acquired in the text processing, is the norm. Such systems rarely exceed the performance of any single modality (i.e. text or video) in information retrieval tasks. Our experiments indicate that allowing a rich interaction between the modalities results in significant improvement in performance over any single modality. We demonstrate these results using the TRECVID03 corpus, which comprises 120 hours of broadcast news videos. Our results demonstrate over 14 % improvement in IR performance over the best reported text-only baseline and ranks amongst the best results reported on this corpus.
50 citations
"Learning ontology for personalized ..." refers background in this paper
...The requirement is more pronounced for multimedia retrieval systems since interpretation of media data is highly subjective and contextual....
[...]
13 Jul 2006
TL;DR: A novel approach for visual scene representation, combining the use of quantized color and texture local invariant features (referred to here as visterms) computed over interest point regions, by investigating the different ways to fuse together local information from texture and color in order to provide a better visterm representation.
Abstract: This paper presents a novel approach for visual scene representation, combining the use of quantized color and texture local invariant features (referred to here as visterms) computed over interest point regions. In particular we investigate the different ways to fuse together local information from texture and color in order to provide a better visterm representation. We develop and test our methods on the task of image classification using a 6-class natural scene database. We perform classification based on the bag-of-visterms (BOV) representation (histogram of quantized local descriptors), extracted from both texture and color features. We investigate two different fusion approaches at the feature level: fusing local descriptors together and creating one representation of joint texture-color visterms, or concatenating the histogram representation of both color and texture, obtained independently from each local feature. On our classification task we show that the appropriate use of color improves the results w.r.t. a texture only representation.
40 citations
"Learning ontology for personalized ..." refers background in this paper
...Thus, a retrieval system with a .xed ranking algorithm can never satisfy all its users....
[...]