Within the SemEval-2013 evaluation exercise, the TempEval-3 shared task aims to advance research on temporal information processing. It follows on from TempEval-1 and -2, with: a three-part structure covering temporal expression, event, and temporal relation extraction; a larger dataset; and new single measures to rank systems ‐ in each task and in general. In this paper, we describe the participants’ approaches, results, and the observations from the results, which may guide future research in this area.

/pdf/semeval-2013-task-1-tempeval-3-evaluating-time-expressions-cs89sl31fr.pdf

SemEval-2013 Task 1: TempEval-3: Evaluating Time Expressions, Events, and Temporal Relations

Extraction and normalization of temporal expressions from documents are important steps towards deep text understanding and a prerequisite for many NLP tasks such as information extraction, question answering, and document summarization. There are different ways to express (the same) temporal information in documents. However, after identifying temporal expressions, they can be normalized according to some standard format. This allows the usage of temporal information in a term- and language-independent way. In this paper, we describe the challenges of temporal tagging in different domains, give an overview of existing annotated corpora, and survey existing approaches for temporal tagging. Finally, we present our publicly available temporal tagger HeidelTime, which is easily extensible to further languages due to its strict separation of source code and language resources like patterns and rules. We present a broad evaluation on multiple languages and domains on existing corpora as well as on a newly created corpus for a language/domain combination for which no annotated corpus has been available so far.

Multilingual and cross-domain temporal tagging

A survey on question answering technology from an information retrieval perspective

We describe the design, implementation, and evaluation of EMBERS, an automated, 24x7 continuous system for forecasting civil unrest across 10 countries of Latin America using open source indicators such as tweets, news sources, blogs, economic indicators, and other data sources. Unlike retrospective studies, EMBERS has been making forecasts into the future since Nov 2012 which have been (and continue to be) evaluated by an independent T&E team (MITRE). Of note, EMBERS has successfully forecast the uptick and downtick of incidents during the June 2013 protests in Brazil. We outline the system architecture of EMBERS, individual models that leverage specific data sources, and a fusion and suppression engine that supports trading off specific evaluation criteria. EMBERS also provides an audit trail interface that enables the investigation of why specific predictions were made along with the data utilized for forecasting. Through numerous evaluations, we demonstrate the superiority of EMBERS over baserate methods and its capability to forecast significant societal happenings.

'Beating the news' with EMBERS: Forecasting Civil Unrest using Open Source Indicators

Question Answering (QA) is undoubtedly a growing field of current research in Artificial Intelligence. Question classification, a QA subtask, aims to associate a category to each question, typically representing the semantic class of its answer. This step is of major importance in the QA process, since it is the basis of several key decisions. For instance, classification helps reducing the number of possible answer candidates, as only answers matching the question category should be taken into account. This paper presents and evaluates a rule-based question classifier that partially founds its performance in the detection of the question headword and in its mapping into the target category through the use of WordNet. Moreover, we use the rule-based classifier as a features' provider of a machine learning-based question classifier. A detailed analysis of the rule-base contribution is presented. Despite using a very compact feature space, state of the art results are obtained.

From symbolic to sub-symbolic information in question classification

This paper presents TIPSem, a system to extract temporal information from natural language texts for English and Spanish. TIPSem, learns CRF models from training data. Although the used features include different language analysis levels, the approach is focused on semantic information. For Spanish, TIPSem achieved the best F1 score in all the tasks. For English, it obtained the best F1 in tasks B (events) and D (event-dct links); and was among the best systems in the rest.

TIPSem (English and Spanish): Evaluating CRFs and Semantic Roles in TempEval-2

This paper presents a multi-layered Question Answering (Q.A.) architecture suitable for enhancing current Q.A. capabilities with the possibility of processing complex questions. That is, questions whose answer needs to be gathered from pieces of factual information scattered in different documents. Specifically, we have designed a layer oriented to process the different types of temporal questions. Complex temporal questions are first decomposed into simpler ones, according to the temporal relationships expressed in the original question.In the same way, the answers of each simple question are re-composed, fulfilling the temporal restrictions of the original complex question.Using this architecture, a Temporal Q.A. system has been developed.In this paper, we focus on explaining the first part of the process: the decomposition of the complex questions. Furthermore, it has been evaluated with the TERQAS question corpus of 112 temporal questions. For the task of question splitting our system has performed, in terms of precision and recall, 85% and 71%, respectively.

/pdf/splitting-complex-temporal-questions-for-question-answering-2kwjjmj8rk.pdf

Splitting Complex Temporal Questions for Question Answering Systems

Temporal expressions are words or phrases that describe a point, duration or recurrence in time. Automatically annotating these expressions is a research goal of increasing interest. Recognising them can be achieved with minimally supervised machine learning, but interpreting them accurately (normalisation) is a complex task requiring human knowledge. In this paper, we present TIMEN, a community-driven tool for temporal expression normalisation. TIMEN is derived from current best approaches and is an independent tool, enabling easy integration in existing systems. We argue that temporal expression normalisation can only be effectively performed with a large knowledge base and set of rules. Our solution is a framework and system with which to capture this knowledge for different languages. Using both existing and newly-annotated data, we present results showing competitive performance and invite the IE community to contribute to a knowledge base in order to solve the temporal expression normalisation problem.

/pdf/timen-an-open-temporal-expression-normalisation-resource-qqa0w9er4v.pdf

TIMEN: An Open Temporal Expression Normalisation Resource

Post-truth is a term that describes a distorting phenomenon that aims to manipulate public opinion and behavior. One of its key engines is the spread of Fake News. Nowadays most news is rapidly disseminated in written language via digital media and social networks. Therefore, to detect fake news it is becoming increasingly necessary to apply Artificial Intelligence (AI) and, more specifically Natural Language Processing (NLP). This paper presents a review of the application of AI to the complex task of automatically detecting fake news. The review begins with a definition and classification of fake news. Considering the complexity of the fake news detection task, a divide-and-conquer methodology was applied to identify a series of subtasks to tackle the problem from a computational perspective. As a result, the following subtasks were identified: deception detection; stance detection; controversy and polarization; automated fact checking; clickbait detection; and, credibility scores. From each subtask, a PRISMA compliant systematic review of the main studies was undertaken, searching Google Scholar. The various approaches and technologies are surveyed, as well as the resources and competitions that have been involved in resolving the different subtasks. The review concludes with a roadmap for addressing the future challenges that have emerged from the analysis of the state of the art, providing a rich source of potential work for the research community going forward.

Fighting post-truth using natural language processing: A review and open challenges

This paper presents two proposals based on semantic information, semantic roles and WordNet, for the answer extraction module of a general open-domain question answering (QA) system. The main objective of this research is to determine how the system performance is influenced by using this kind of information, and compare it with that of current QA systems based on named entities (NEs). NE-based QA systems achieve good results with NE-based questions. However, with common noun (CN) based questions, like ''Where is the stomach? In the abdomen'', they fail, and this is the main reason for our study. In this paper our new proposals for answering different types of questions are evaluated and compared with an NE-based approach for both NE-based and CN-based questions. From the results obtained it may be concluded that, with the aid of our proposals, the QA system performs much better with CN-based questions when semantic information is used (semantic information F"@b"="1=74.73% vs. NEF"@b"="1=12.19%). Moreover, the more semantic information the system uses, the better the precision and correctness of the answer it achieves.

Estela Saquete

Papers

TIPSem (English and Spanish): Evaluating CRFs and Semantic Roles in TempEval-2

Splitting Complex Temporal Questions for Question Answering Systems

TIMEN: An Open Temporal Expression Normalisation Resource

Fighting post-truth using natural language processing: A review and open challenges

Combining semantic information in question answering systems