We present YAGO, a light-weight and extensible ontology with high coverage and quality. YAGO builds on entities and relations and currently contains more than 1 million entities and 5 million facts. This includes the Is-A hierarchy as well as non-taxonomic relations between entities (such as HASONEPRIZE). The facts have been automatically extracted from Wikipedia and unified with WordNet, using a carefully designed combination of rule-based and heuristic methods described in this paper. The resulting knowledge base is a major step beyond WordNet: in quality by adding knowledge about individuals like persons, organizations, products, etc. with their semantic relationships - and in quantity by increasing the number of facts by more than an order of magnitude. Our empirical evaluation of fact correctness shows an accuracy of about 95%. YAGO is based on a logically clean model, which is decidable, extensible, and compatible with RDFS. Finally, we show how YAGO can be further extended by state-of-the-art information extraction techniques.

/pdf/yago-a-core-of-semantic-knowledge-13k3nmj6l2.pdf

Yago: a core of semantic knowledge

Originally published in Contemporary Psychology: APA Review of Books, 1994, Vol 39(2), 225. Reviews the book, The Nature and Origins of Mass Opinion by John Zaller (1992). The author's commendable effort to specify a model of mass opinion formation offers readers an introduction to the prevailing vi

The Nature and Origins of Mass Opinion.

Modern models of relation extraction for tasks like ACE are based on supervised learning of relations from small hand-labeled corpora. We investigate an alternative paradigm that does not require labeled corpora, avoiding the domain dependence of ACE-style algorithms, and allowing the use of corpora of any size. Our experiments use Freebase, a large semantic database of several thousand relations, to provide distant supervision. For each pair of entities that appears in some Freebase relation, we find all sentences containing those entities in a large unlabeled corpus and extract textual features to train a relation classifier. Our algorithm combines the advantages of supervised IE (combining 400,000 noisy pattern features in a probabilistic classifier) and unsupervised IE (extracting large numbers of relations from large corpora of any domain). Our model is able to extract 10,000 instances of 102 relations at a precision of 67.6%. We also analyze feature performance, showing that syntactic parse features are particularly helpful for relations that are ambiguous or lexically distant in their expression.

/pdf/distant-supervision-for-relation-extraction-without-labeled-2nqml9l790.pdf

Distant supervision for relation extraction without labeled data

This survey covers fifteen years of research in the Named Entity Recognition and Classification (NERC) field, from 1991 to 2006. We report observations about languages, named entity types, domains and textual genres studied in the literature. From the start, NERC systems have been developed using hand-made rules, but now machine learning techniques are widely used. These techniques are surveyed along with other critical aspects of NERC such as features and evaluation methods. Features are word-level, dictionary-level and corpus-level representations of words in a document. Evaluation techniques, ranging from intuitive exact match to very complex matching techniques with adjustable cost of errors, are an indisputable key to progress.

A survey of named entity recognition and classification

This tutorial is concerned with a comprehensive introduction to the research area of learning to rank for information retrieval. In the first part of the tutorial, we will introduce three major approaches to learning to rank, i.e., the pointwise, pairwise, and listwise approaches, analyze the relationship between the loss functions used in these approaches and the widely-used IR evaluation measures, evaluate the performance of these approaches on the LETOR benchmark datasets, and demonstrate how to use these approaches to solve real ranking applications. In the second part of the tutorial, we will discuss some advanced topics regarding learning to rank, such as relational ranking, diverse ranking, semi-supervised ranking, transfer ranking, query-dependent ranking, and training data preprocessing. In the third part, we will briefly mention the recent advances on statistical learning theory for ranking, which explain the generalization ability and statistical consistency of different ranking methods. In the last part, we will conclude the tutorial and show several future research directions.

/pdf/learning-to-rank-for-information-retrieval-2q6t3viyug.pdf

Learning to Rank for Information Retrieval

Automatic question answering is a central topic in information retrieval. Web search engines have made great progress at answering factoid queries, such as “how many people live in Australia?”. These can provide a succinct answer, up to a few words in length, and sometimes offer additional information such as related facts or entities. However, for deeper questions which could benefit from a longer response (e.g., “history of Australia”), current search engine resort to returning a link to a detailed web document. Alternatively, such a question might be posted on a Community Question Answering (CQA) site (“Visiting Australia in May, what should I see?”), hoping to get a human authored and detailed response. In this workshop we aim to explore the boundaries of Web question answering to better understand the spectrum of approaches and possible responses that are more detailed than a short fact, yet are more useful than a full document result. Is it possible to automatically answer diverse questions ranging from advice on fixing a broken sink to requests for opinions on the best basketball player of all time. In addition, questions submitted on the Web can be either short and ambiguous (such as Web queries to a search engine), or long and detailed (such as CQA questions). This workshop is particularly timely for two additional reasons: (1) there still exist many disagreements regarding the goals and nature of Web question answering services, mostly relating to the questions of “question intent” (what kind of queries benefit from question answering compared to other methods); and (2) leading search engines are eager to provide

Web Question Answering: Beyond Factoids: SIGIR 2015 Workshop

It is our great pleasure to welcome you to the workshop on Search in Social Media (SSM 2008), co-located with CIKM 2008. The purpose of this workshop is to bring together academic and industry researchers in information retrieval and social media to consider the following questions: How should we search in social media? What are the needs of users, and models of those needs, specific to social media search? What models make the most sense? How does search interact with existing uses of social media? What works and what does not?

The call for papers attracted 22 submissions, of which 10 papers were accepted as regular papers, and four papers were accepted as position papers. In addition, the program includes a panel on the future of search in social media, and a keynote speech by Andrew Tomkins, Chief Scientist of Search at Yahoo! on Challenges in Searching Social Media.

Proceedings of the 2008 ACM workshop on Search in social media

Users' clicks on Web search results are one of the key signals for evaluating and improving web search quality and have been widely used as part of current state-of-the-art Learning-To-Rank(LTR) models. With a large volume of search logs available for major search engines, effective models of searcher click behavior have emerged to evaluate and train LTR models. However, when modeling the users' click behavior, considering the bias of the behavior is imperative. In particular, when a search result is not clicked, it is not necessarily chosen as not relevant by the user, but instead could have been simply missed, especially for lower-ranked results. These kinds of biases in the click log data can be incorporated into the click models, propagating the errors to the resulting LTR ranking models or evaluation metrics. In this paper, we propose the De-biased Reinforcement Learning Click model (DRLC). The DRLC model relaxes previously made assumptions about the users' examination behavior and resulting latent states. To implement the DRLC model, convolutional neural networks are used as the value networks for reinforcement learning, trained to learn a policy to reduce bias in the click logs. To demonstrate the effectiveness of the DRLC model, we first compare performance with the previous state-of-art approaches using established click prediction metrics, including log-likelihood and perplexity. We further show that DRLC also leads to improvements in ranking performance. Our experiments demonstrate the effectiveness of the DRLC model in learning to reduce bias in click logs, leading to improved modeling performance and showing the potential for using DRLC for improving Web search quality.

https://dl.acm.org/doi/pdf/10.1145/3404835.3463228

De-Biased Modeling of Search Click Behavior with Reinforcement Learning

This workshop brings together researchers and practitioners from industry and academia to discuss search and discovery in the medi-cal domain. The event focuses on ways to make medical and health information more accessible to laypeople (including enhancements to ranking algorithms and search interfaces), and how we can dis-cover new medical facts and phenomena from information sought online, as evidenced in query streams and other sources such as social media. This domain also offers many opportunities for appli-cations that monitor and improve quality of life of those affected by medical conditions, by providing tools to support their health-related information behavior.

Workshop on health search and discovery: helping users and advancing medicine

Most of the information on the Web is inherently structured, product pages of large online shopping sites such as Amazon.com being a typical example. Yet, unstructured keyword queries are still the most common way to search for such structured information, producing an ambiguities and poor ranking, and by that degrading user experience. This problem can be resolved by query segmentation, that is, transformation of unstructured keyword queries into structured queries. The resulting queries can be used to search product databases more accurately, and improve result presentation and query suggestion. The main contribution of our work is a novel approach to query segmentation based on unsupervised machine learning. Its highlight is that query and click-through logs are used for training. Extensive experiments over a large query and click log from a leading shopping engine demonstrate that our approach significantly outperforms baseline.

Eugene Agichtein

Papers

Web Question Answering: Beyond Factoids: SIGIR 2015 Workshop

Proceedings of the 2008 ACM workshop on Search in social media

De-Biased Modeling of Search Click Behavior with Reinforcement Learning

Workshop on health search and discovery: helping users and advancing medicine

Mining query structure from click data: a case study of product queries