Topic
Ranking (information retrieval)
About: Ranking (information retrieval) is a research topic. Over the lifetime, 21109 publications have been published within this topic receiving 435130 citations.
Papers published on a yearly basis
Papers
More filters
••
25 Jul 2020TL;DR: Zhang et al. as mentioned in this paper introduced an open-retrieval conversational question answering (ORConvQA) setting, where they learn to retrieve evidence from a large collection before extracting answers, as a further step towards building functional conversational search systems.
Abstract: Conversational search is one of the ultimate goals of information retrieval. Recent research approaches conversational search by simplified settings of response ranking and conversational question answering, where an answer is either selected from a given candidate set or extracted from a given passage. These simplifications neglect the fundamental role of retrieval in conversational search. To address this limitation, we introduce an open-retrieval conversational question answering (ORConvQA) setting, where we learn to retrieve evidence from a large collection before extracting answers, as a further step towards building functional conversational search systems. We create a dataset, OR-QuAC, to facilitate research on ORConvQA. We build an end-to-end system for ORConvQA, featuring a retriever, a reranker, and a reader that are all based on Transformers. Our extensive experiments on OR-QuAC demonstrate that a learnable retriever is crucial for ORConvQA. We further show that our system can make a substantial improvement when we enable history modeling in all system components. Moreover, we show that the reranker component contributes to the model performance by providing a regularization effect. Finally, further in-depth analyses are performed to provide new insights into ORConvQA.
108 citations
••
NEC1
TL;DR: The notion of bookmark management is extended by introducing the functionalities of hypermedia databases, and PowerBookmarks supports advanced query, classification, and navigation functionalities on collections of bookmarks.
Abstract: We extend the notion of bookmark management by introducing the functionalities of hypermedia databases. PowerBookmarks is a Web information organization, sharing, and management tool, which parses metadata from bookmarked URLs and uses it to index and classify the URLs. PowerBookmarks supports advanced query, classification, and navigation functionalities on collections of bookmarks. PowerBookmarks monitors and utilizes users' access patterns to provide many useful personalized services, such as automated URL bookmarking, document refreshing, and bookmark expiration. It also allows users to specify their preference in bookmark management, such as ranking schemes and classification tree structures. Subscription services for new or updated documents of users' interests are also supported.
108 citations
••
30 Oct 2008TL;DR: A machine learning approach for retrieving sounds that is novel in that it uses free-form text queries rather sound sample based queries, searches by audio content rather than via textual meta data, and can scale to very large number of audio documents and very rich query vocabulary.
Abstract: In content-based audio retrieval, the goal is to find sound recordings (audio documents) based on their acoustic features. This content-based approach differs from retrieval approaches that index media files using metadata such as file names and user tags. In this paper, we propose a machine learning approach for retrieving sounds that is novel in that it (1) uses free-form text queries rather sound sample based queries, (2) searches by audio content rather than via textual meta data, and (3) can scale to very large number of audio documents and very rich query vocabulary. We handle generic sounds, including a wide variety of sound effects, animal vocalizations and natural scenes. We test a scalable approach based on a passive-aggressive model for image retrieval (PAMIR), and compare it to two state-of-the-art approaches; Gaussian mixture models (GMM) and support vector machines (SVM).We test our approach on two large real-world datasets: a collection of short sound effects, and a noisier and larger collection of user-contributed user-labeled recordings (25K files, 2000 terms vocabulary). We find that all three methods achieved very good retrieval performance. For instance, a positive document is retrieved in the first position of the ranking more than half the time, and on average there are more than 4 positive documents in the first 10 retrieved, for both datasets. PAMIR completed both training and retrieval of all data in less than 6 hours for both datasets, on a single machine. It was one to three orders of magnitude faster than the competing approaches. This approach should therefore scale to much larger datasets in the future.
108 citations
•
IBM1
TL;DR: In this article, a ranking function is applied to each of the candidate answers to determine a ranking for each of these candidate answers; and for each subquery, one of the candidates answers to the subquery is selected based on this ranking.
Abstract: A method, system and computer program product for generating answers to questions. In one embodiment, the method comprises receiving an input query, decomposing the input query into a plurality of different subqueries, and conducting a search in one or more data sources to identify at least one candidate answer to each of the subqueries. A ranking function is applied to each of the candidate answers to determine a ranking for each of these candidate answers; and for each of the subqueries, one of the candidate answers to the subquery is selected based on this ranking. A logical synthesis component is applied to synthesize a candidate answer for the input query from the selected the candidate answers to the subqueries. In one embodiment, the procedure applied by the logical synthesis component to synthesize the candidate answer for the input query is determined from the input query.
108 citations
••
18 Aug 2010TL;DR: The thesis investigates how the observed quality of predictors affects the retrieval effectiveness in two adaptive system settings: selective query expansion and meta-search and provides an analysis of its sensitivity towards different variables such as the collection, the query set and the retrieval approach.
Abstract: We consider users' attempts to express their information needs through queries, or search requests and try to predict whether those requests will be of high or low quality. The second type of methods under investigation are those which attempt to estimate the quality of search systems themselves. Given a number of search systems to consider, these methods estimate how well or how poorly the systems will perform in comparison to each other.First, pre-retrieval predictors are investigated, which predict a query's effectiveness before the retrieval step and are thus independent of the ranked list of results. Such predictors base their predictions solely on query terms, collection statistics and possibly external sources. Twenty-two prediction algorithms are categorized and their quality is assessed on three different TREC test collections. A number of newly applied methods for combining various predictors are examined to obtain a better prediction of a query's effectiveness.Building on the analysis of pre-retrieval predictors, post-retrieval approaches are then investigated, which estimate a query's effectiveness on the basis of the retrieved results. The thesis focuses in particular on the Clarity Score approach and provides an analysis of its sensitivity towards different variables such as the collection, the query set and the retrieval approach. Adaptations to Clarity Score are introduced which improve the estimation accuracy of the original algorithm.The utility of query effectiveness prediction methods is commonly evaluated by reporting correlation coefficients, such as Kendall's Tau. Largely unexplored though is the question of the relationship between the current evaluation methodology for query effectiveness prediction and the change in effectiveness of retrieval systems that employ a predictor. We investigate this question by examining how the observed quality of predictors (with respect to Kendall's Tau) affects the retrieval effectiveness in two adaptive system settings: selective query expansion and meta-search.The last part of the thesis is concerned with the task of estimating the ranking of retrieval systems according to their retrieval effectiveness without relying on costly relevance judgments. Five different system ranking estimation approaches are evaluated on a wide range of data sets which cover a variety of retrieval tasks and test collections. It is shown that under certain conditions, automatic methods yield a highly accurate ranking of systems.Available online at http://www.cs.utwente.nl/~hauffc/phd/thesis.pdf.
107 citations