Document retrieval

About: Document retrieval is a research topic. Over the lifetime, 6821 publications have been published within this topic receiving 214383 citations.

Probabilistic Retrieval of OCR Degraded Text Using N-Grams

Abstract: The retrieval of OCR degraded text using n-gram formulations within a probabilistic retrieval system is examined in this paper. Direct retrieval of documents using n-gram databases of 2 and 3-grams or 2, 3, 4 and 5-grams resulted in improved retrieval performance over standard (word based) queries on the same data when a level of 10 percent degradation or worse was achieved. A second method of using n-grams to identify appropriate matching and near matching terms for query expansion which also performed better than using standard queries is also described. This method was less effective than direct n-gram query formulations but can likely be improved with alternative query component weighting schemes and measures of term similarity. Finally, a web based retrieval application using n-gram retrieval of OCR text and display, with query term highlighting, of the source document image is described.

Unified utility maximization framework for resource selection

Abstract: This paper presents a unified utility framework for resource selection of distributed text information retrieval. This new framework shows an efficient and effective way to infer the probabilities of relevance of all the documents across the text databases. With the estimated relevance information, resource selection can be made by explicitly optimizing the goals of different applications. Specifically, when used for database recommendation, the selection is optimized for the goal of high-recall (include as many relevant documents as possible in the selected databases); when used for distributed document retrieval, the selection targets the high-precision goal (high precision in the final merged list of documents). This new model provides a more solid framework for distributed information retrieval. Empirical studies show that it is at least as effective as other state-of-the-art algorithms.

A model of knowledge based information retrieval with hierarchical concept

Abstract: This paper discusses a knowledge based information retrieval model with hierarchical thesaurus. The model computes the conceptual distance between a query and an object and both are indexed with weighted terms from a hierarchical thesaurus. The hierarchical thesaurus is represented by a hierarchical‐concept graph (HCG) in which nodes represent concepts and directed edges represent generalisation relationships. Rada et al. have developed a similar model. However, their model considered only a binary indexing scheme and revealed some counter‐intuitive results. Our proposed model extends theirs by allowing the index term and the edge of the HCG to be weighted. A new concept mapping method is devised to overcome Rada's counter‐intuitive results. In addition, a scheme for allowing Boolean operators in user queries is provided with a formula for computing conceptual distance from negated index terms. Experimental results have shown that our model simulates human performance more closely than Rada's model.

A machine learning approach for improved BM25 retrieval

Abstract: Despite the widespread use of BM25, there have been few studies examining its effectiveness on a document description over single and multiple field combinations. We determine the effectiveness of BM25 on various document fields. We find that BM25 models relevance on popularity fields such as anchor text and query click information no better than a linear function of the field attributes. We also find query click information to be the single most important field for retrieval. In response, we develop a machine learning approach to BM25-style retrieval that learns, using LambdaRank, from the input attributes of BM25. Our model significantly improves retrieval effectiveness over BM25 and BM25F. Our data-driven approach is fast, effective, avoids the problem of parameter tuning, and can directly optimize for several common information retrieval measures. We demonstrate the advantages of our model on a very large real-world Web data collection.