We present YAGO, a light-weight and extensible ontology with high coverage and quality. YAGO builds on entities and relations and currently contains more than 1 million entities and 5 million facts. This includes the Is-A hierarchy as well as non-taxonomic relations between entities (such as HASONEPRIZE). The facts have been automatically extracted from Wikipedia and unified with WordNet, using a carefully designed combination of rule-based and heuristic methods described in this paper. The resulting knowledge base is a major step beyond WordNet: in quality by adding knowledge about individuals like persons, organizations, products, etc. with their semantic relationships - and in quantity by increasing the number of facts by more than an order of magnitude. Our empirical evaluation of fact correctness shows an accuracy of about 95%. YAGO is based on a logically clean model, which is decidable, extensible, and compatible with RDFS. Finally, we show how YAGO can be further extended by state-of-the-art information extraction techniques.

/pdf/yago-a-core-of-semantic-knowledge-13k3nmj6l2.pdf

Yago: a core of semantic knowledge

Originally published in Contemporary Psychology: APA Review of Books, 1994, Vol 39(2), 225. Reviews the book, The Nature and Origins of Mass Opinion by John Zaller (1992). The author's commendable effort to specify a model of mass opinion formation offers readers an introduction to the prevailing vi

The Nature and Origins of Mass Opinion.

Modern models of relation extraction for tasks like ACE are based on supervised learning of relations from small hand-labeled corpora. We investigate an alternative paradigm that does not require labeled corpora, avoiding the domain dependence of ACE-style algorithms, and allowing the use of corpora of any size. Our experiments use Freebase, a large semantic database of several thousand relations, to provide distant supervision. For each pair of entities that appears in some Freebase relation, we find all sentences containing those entities in a large unlabeled corpus and extract textual features to train a relation classifier. Our algorithm combines the advantages of supervised IE (combining 400,000 noisy pattern features in a probabilistic classifier) and unsupervised IE (extracting large numbers of relations from large corpora of any domain). Our model is able to extract 10,000 instances of 102 relations at a precision of 67.6%. We also analyze feature performance, showing that syntactic parse features are particularly helpful for relations that are ambiguous or lexically distant in their expression.

/pdf/distant-supervision-for-relation-extraction-without-labeled-2nqml9l790.pdf

Distant supervision for relation extraction without labeled data

This survey covers fifteen years of research in the Named Entity Recognition and Classification (NERC) field, from 1991 to 2006. We report observations about languages, named entity types, domains and textual genres studied in the literature. From the start, NERC systems have been developed using hand-made rules, but now machine learning techniques are widely used. These techniques are surveyed along with other critical aspects of NERC such as features and evaluation methods. Features are word-level, dictionary-level and corpus-level representations of words in a document. Evaluation techniques, ranging from intuitive exact match to very complex matching techniques with adjustable cost of errors, are an indisputable key to progress.

A survey of named entity recognition and classification

This tutorial is concerned with a comprehensive introduction to the research area of learning to rank for information retrieval. In the first part of the tutorial, we will introduce three major approaches to learning to rank, i.e., the pointwise, pairwise, and listwise approaches, analyze the relationship between the loss functions used in these approaches and the widely-used IR evaluation measures, evaluate the performance of these approaches on the LETOR benchmark datasets, and demonstrate how to use these approaches to solve real ranking applications. In the second part of the tutorial, we will discuss some advanced topics regarding learning to rank, such as relational ranking, diverse ranking, semi-supervised ranking, transfer ranking, query-dependent ranking, and training data preprocessing. In the third part, we will briefly mention the recent advances on statistical learning theory for ranking, which explain the generalization ability and statistical consistency of different ranking methods. In the last part, we will conclude the tutorial and show several future research directions.

/pdf/learning-to-rank-for-information-retrieval-2q6t3viyug.pdf

Learning to Rank for Information Retrieval

Passage retrieval is a crucial first step of automatic Question Answering (QA). While existing passage retrieval algorithms are effective at selecting document passages most similar to the question, or those that contain the expected answer types, they do not take into account which parts of the document the searchers actually found useful. We propose, to the best of our knowledge, the first successful attempt to incorporate searcher examination data into passage retrieval for question answering. Specifically, we exploit detailed examination data, such as mouse cursor movements and scrolling, to infer the parts of the document the searcher found interesting, and then incorporate this signal into passage retrieval for QA. Our extensive experiments and analysis demonstrate that our method significantly improves passage retrieval, compared to using textual features alone. As an additional contribution, we make available to the research community the code and the search behavior data used in this study, with the hope of encouraging further research in this area.

The Answer is at your Fingertips: Improving Passage Retrieval for Web Question Answering with Search Behavior Data

In this paper, we highlight the potential of n-grams as a vehicle to explore the ‘evolution’ of the law and legal language. Using the full text corpus of decisions of the United States Supreme Court (1791-2005), we explore the n-gram space, offer some initial results based upon our calculations and highlight the beta version of our n-gram search interface.

https://papers.ssrn.com/sol3/Delivery.cfm?abstractid=1971953

Legal N-Grams? A Simple Approach to Track the ‘Evolution’ of Legal Language

Effective search session segmentation "grouping queries according to common task or intent" can be useful for improving relevance, search evaluation, and query suggestion. Previous work has largely attempted to segment search sessions off-line, after the fact. In contrast, we present preliminary investigation of predicting, in real time, whether a user is about to switch interest - that is, whether the user is about to finish the current search and switch to another search task (or stop searching altogether). We explore an approach for this task using client-side user behavior such as clicks, scrolls, and mouse movements, contextualized by the content of the search result pages and previous searches. Our experiments over thousands of real searches show that we can identify context and user behavior patterns that indicate that a user is about to switch to a new search task. These preliminary results can be helpful for more effective query suggestion and personalization.

Beyond session segmentation: predicting changes in search intent with client-side user interactions

Background: A wealth of information is hidden within unstructured text. This information is often best utilized in structured or relational form, which is suited for sophisticated query processing, for integration with relational databases, and for data mining. For example, newspaper and e-mail archives contain information that could be useful to analysts and govenment agencies. Information extraction systems produce a structured representation of the information that is “buried” in text documents. Unfortunately, processing each document is computationally expensive, and is not feasible for large text databases or for the web. With many database sizes exceeding millions of documents, processing time is becoming a bottleneck for exploiting information extraction technology.

QXtract: a building block for efficient information extraction from text databases

Semantically similar questions are submitted to collaborative question answering systems repeatedly even though these questions already contain best answers before. To solve the problem, we propose a precise approach of automatically finding an answer to such questions by identifying "equivalent" questions submitted and answered. Our method is based on a new pattern generation method T-IPG to automatically extract equivalent question patterns. Taking these patterns from training data as seed patterns, we further propose a bootstrap-based pattern learning method to extend more equivalent patterns on these seed patterns. The resulting patterns can be applied to match a new question to an equivalent one that has already been answered, and thus suggest potential answers automatically. We experimented with this approach over a large collection of more than 200,000 real questions drawn from Yahoo! Answers archive, automatically acquiring over 16,991 equivalent question patterns. These patterns allow our method to obtain over 57% recall and over 54% precision on suggesting an answer automatically to new questions, significantly improving over baseline methods.

Eugene Agichtein

Papers

The Answer is at your Fingertips: Improving Passage Retrieval for Web Question Answering with Search Behavior Data

Legal N-Grams? A Simple Approach to Track the ‘Evolution’ of Legal Language

Beyond session segmentation: predicting changes in search intent with client-side user interactions

QXtract: a building block for efficient information extraction from text databases

Bootstrap-Based equivalent pattern learning for collaborative question answering