There has recently been an increased interest in named entity recognition and disambiguation systems at major conferences such as WWW, SIGIR, ACL, KDD, etc. However, most work has focused on algorithms and evaluations, leaving little space for implementation details. In this paper, we discuss some implementation and data processing challenges we encountered while developing a new multilingual version of DBpedia Spotlight that is faster, more accurate and easier to configure. We compare our solution to the previous system, considering time performance, space requirements and accuracy in the context of the Dutch and English languages. Additionally, we report results for 9 additional languages among the largest Wikipedias. Finally, we present challenges and experiences to foment the discussion with other developers interested in recognition and disambiguation of entities in natural language text.

Improving efficiency and accuracy in multilingual entity extraction

We present a system for the detection of the stance of headlines with regard to their corresponding article bodies The approach can be applied in fake news, especially clickbait detection scenarios The component is part of a larger platform for the curation of digital content; we consider veracity and relevancy an increasingly important part of curating online information We want to contribute to the debate on how to deal with fake news and related online phenomena with technological means, by providing means to separate related from unrelated headlines and further classifying the related headlines On a publicly available data set annotated for the stance of headlines with regard to their corresponding article bodies, we achieve a (weighted) accuracy score of 8959

/pdf/from-clickbait-to-fake-news-detection-an-approach-based-on-58q05qsy1s.pdf

From Clickbait to Fake News Detection: An Approach based on Detecting the Stance of Headlines to Articles

Named Entity Disambiguation is the task of assigning entities from a Knowledge Base to mentions of such entities in a textual document. This article presents two novel approaches guided by a natural notion of semantic similarity for the collective disambiguation of all entities mentioned in a document at the same time. We adopt a unified semantic representation for entities and documents—the probability distribution obtained from a random walk on a subgraph of the knowledge base—as well as lexical and statistical features commonly used for this task. The first approach is an iterative and greedy approximation, while the second is based on learning-to-rank. Our experimental evaluation uses well-known benchmarks as well as new ones introduced here. We justify the need for these new benchmarks, and show that both our methods outperform the previous state-of-the-art by a wide margin.

Robust named entity disambiguation with random walks

This paper deals with the representation of multi-level linguistic annotations. It proposes an XML-based, generic stand-off architecture and presents an example instantiation. Application scenarios that profit from this architecture are sketched out. In recent years, corpus linguistics has become more and more important to a broad community, including people working in theoretical, applied and computational linguistics. To many of them, speech and text corpora represent a rich source of data and phenomena, forming the basis of their research. Benefit from such data is even more important if the data is annotated by suitable information, allowing for fast and effective retrieval of relevant data. Whereas corpora of the first generation featured part-of-speech and syntactic annotations (e.g. PennTreebank [MSM93], TIGER corpus [BDE04]), the focus has now switched to properties beyond the (morpho-)syntactic level. Recent corpora are annotated by semantic information (PropBank [KP02], FrameNet [JPB03], SALSA [EKPP03]), pragmatic information (Penn Discourse TreeBank [MPJW04], RST Discourse Treebank [CMO03], Potsdam Commentary Corpus [Ste04]), and dialogue structure (Switchboard SWBD-DAMSL [JSB97]). Annotations often have to be carried out manually — reliable (semi-)automatic tools exist only for the annotation of part of speech and syntax, and are restricted to well-researched languages like English or German. Moreover, hand-annotated training material is a prerequisite for the development of automatic tools. As a consequence, corpora and annotations ought to be reusable so that a large community can profit from the data. To this end, various standardization efforts have been launched. Standardization of linguistic data concerns (see, e.g., [Sch05]): (i) The physical data structure: here, XML has become the widely-recognized standard format. (ii) The logical data structure: i.e., the data models that are used to model the phenomena and their properties (e.g. hierarchical structures like trees or graphs for syntax annotations 1The research reported in this paper was jointly financed by the German Research Foundation (DFG, SFB632) and the Federal Ministry of Education and Research (BMBF grant no. 03WKH22). Many thanks go to my colleagues, especially Michael Götze, for helpful discussions of the topics addressed in this paper. vs. time-aligned tiers for speech and dialogue annotations). Examples of data models are annotation graphs [BL01] and the NITE Object Model [CKO03b]. (iii) Content: in several initiatives, XML applications for specific linguistic annotations have been developed. For instance, TEI2 (“Text Encoding Initiative”, [SB94]) defines highly-detailed DTDs for encoding all kinds of bibliographic and other information; XCES3 (“XML-based Corpus Encoding Standard”) provides DTDs for the annotation of chunks, alignment, etc. More recently, however, it has been recognized that these standardized DTDs often do not meet application-specific needs. Hence, abstract, generic XML formats have been proposed that allow for the formal integration of application-specific annotations [IR01]. For the conceptual integration of specific annotations, so-called data category repositories as well as linguistic ontologies have been developed. They define reference categories, with precise semantics and examples, that specific annotation tags ought to be mapped to (see, e.g., DOLCE4, “Descriptive Ontology for Linguistic and Cognitive Engineering”). This papers deals with the formal integration of specific annotations. It first addresses the subject of stand-off architecture (sec. 1). We then propose an XML-based representation of linguistic annotation and present an example application (instantiation) in some detail (sec. 2). We also sketch out some application scenarios that profit from such a flexible architecture (sec. 3) and address related approaches (sec. 4). 1 Stand-off Architecture As early as in the mid-nineties, the topic of “stand-off annotation” has been discussed (see, e.g., [TM97]). This term describes the situation where primary data (e.g., the source text) and annotations of this data are stored in separate files. Stand-off annotation might seem problematic, because there is no immediate connection between the text and its annotation; hence, whenever the source text is modified, extra care has to be taken to synchronize its annotation. Similarly, human inspection of the data becomes cumbersome. On the other hand, however, stand-off annotation has the great advantage of leaving the source text untouched. It thus allows for annotating text that cannot be modified for whatever reasons, e.g., because it is a text available on the Internet. Moreover, whereas XML as such does not easily account for overlapping segments and conflicting hierarchies,5 they can be marked in a natural way in stand-off annotation: by distributing annotations over different files. That is, not only is the source text separated from its annotations, but individual annotations are separated from each other as well. This way, annotations at different levels can be created and modified independently of each other. Finally, competing, alternative annotations can even be represented, e.g. variants of part-of-speech annotations that are output of different tools. 2http://www.tei-c.org/ 3http://www.cs.vassar.edu/XCES/ 4http://www.loa-cnr.it/DOLCE.html 5Different methods have been proposed to accommodate conflicting markup into XML. We will come back to them below. One of the first proposals for stand-off annotation of linguistic corpora is [DBD98]. An ISO working group is currently developing the stand-off based LAF6 (“Linguistic Annotation Framework” [IRdlC03]). Some recent corpora like the ANC (“American National Corpus” [RI04]) are encoded in stand-off architecture. In our approach presented in this paper, we also subscribe to the principles of stand-off annotation. 2 A Generic XML Format Our format defines generic XML elements like <mark> (markable), <feat> (feature), and <struct> (structure), which indicate which data type the annotation conforms to. We assume that primary data is stored in a file that optionally specifies a header, followed by a tag <body>, which contains the source text. Annotations are stored in separate files; they may refer to the source text or to other annotations. These relations are encoded by means of XLinks and XPointers. We distinguish three different types of annotations: markables, structures, and features. (i) Markables: <mark> tags specify text positions or spans of text (or spans of other markables) that can be annotated by linguistic information. For instance, <mark> tags might indicate tokens by specifying ranges of the source text, cf. fig. 1. (ii) Structures: <struct> tags are special types of markables. Similar to <mark> tags, they specify objects that then can serve as anchors for annotations. Whereas <mark> tags define simple types of anchors (flat spans of text or markables), a <struct> tag represents a complex anchor involving relations between arbitrarily many markables (including <struct> elements). Relations (<rel>) can be further specified by an attribute type, e.g. as undirected or directed (= pointers). Put differently, a <structList> specifies a complete tree or graph, which consists of single tree fragments specified by the <struct> tags, cf. fig. 1. (iii) Features: <feat> tags specify information annotated to markables or structures, which are referred to by xlink attributes. The type of information (e.g., “part of speech”) is encoded by an attribute type, cf. fig. 2. For instance, the information encoded by the first <feat> in fig. 2 can be paraphrased as follows: Take the token that is defined by the tag <mark> with the ID attribute id="tok 1" and assign the part of speech “ART” (article) to that token. We intend to adopt the idea of [CKO03a] by assuming that admissible feature values (such as “NN”, normal/common noun, or “NE”, named entity) may be complex types and are organized in a type hierarchy. For instance, “NN” and “NE” might be subtypes of the more general type “N”, noun. <feat> tags then point to some type in the hierarchy (which is stored separately), thus specifying the value of the annotated property, cf. fig. 3.7 6ISO Technical TC37/SC4, http://www.tc37sc4.org 7Type hierarchies have to be defined by the user or they may be derived from annotation schemes that incorporate hierarchies, cf. the schemes used by the annotation tool MMAX. In case no hierarchy is defined, the features will be organized in a flat list. The stand-off architecture allows the user to experiment with different hierarchies. Further examples of annotations are sketched out below. They illustrate that annotations may stem from different sources (see the attribute source) and encode various types of information. Categorial annotation (anchored to constituents) <header sfb id="rabin1.const cat" type="categories" source="TIGERcorpus"/> <featList xml:base="rabin1.const.xml"> <feat xlink:href="#syn 1" value="PN"/>  <feat xlink:href="#syn 2" value="PP"/>  ... Coreference annotation, marking coreferential expressions such as pronouns (referred to xlink:href attributes) and their antecedents (identified by target attributes) <header sfb id="rabin1.coref" type="coreference" source="MMAXcoref"/> <featList> <feat xlink:href="rabin1.tok.xml#tok 19" (sein) target="rabin1.const.xml#syn 9" (Der Rabin-Attentäter Jigal Amir) value="identity"/> ... Document structure: headers, paragraphs, lists, etc. (anchored to markables that refer to tokens) <header sfb id="rabin1.div" type="divisions"/> <markList xmlns:xlink="http://www.w3.org/1999/xlink" xml:base="rabin1.tok.xml"> <mark id="div 1" xlink:href="#xpointer(id(’tok 1’)/range-to(id(’tok 390’)"/> <mark id="div 2" xlink:href="#xpointer(id(’tok 1’)/range-to(id(’tok 89’)"/> ... <header sfb id="rabin1.div docstr" type="documentStructure"/> <featL

XML-based Stand-off Representation and Exploitation of Multi-Level Linguistic Annotation.

We present a general framework for integrating annotations from different tools and tag sets. When annotating corpora at multiple linguistic levels, annotators may use different expert tools for different phenomena or types of annotation. These tools employ different data models and accompanying approaches to visualization, and they produce different output formats. For the purposes of uniformly processing these outputs, we developed a pivot format called PAULA, along with converters to and from tool formats. Different annotations are not only integrated at the level of data format, but are also joined on the level of conceptual representation. For this purpose, we introduce OLiA, an ontology of linguistic annotations that mediates between alternative tag sets that cover the same class of linguistic phenomena. All components are integrated in the linguistic information system ANNIS : Annotation tool output is converted to the pivot format PAULA and read into a database where the data can be visualized, queried, and evaluated across multiple layers. For cross-tag set querying and statistical evaluation, ANNIS uses the ontology of linguistic annotations. Finally, ANNIS is also tied to a machine learning component for semiautomatic annotation.

A Flexible Framework for Integrating Annotations from Different Tools and Tag Sets

An approach to the unification of XML (Extensible Markup Language) documents with identical textual content and conc irrent markup in the framework of XML-based multi-layer annotation is introduced. A Prolog program allows the possible relationships between element instances on two annotation layers that share PCDATA to be explored and also the computing of a target node hierarchy for a well-formed, merged XML document. Special attention is paid to identity conflicts between element instances, for which a default solution that takes into account metarelations that hold between element types on the different annotation layers is provided. In addition, rules can be specified by a user to prescribe how identity conflicts should be solved for certain element types.

/pdf/unification-of-xml-documents-with-concurrent-markup-4kabn86puu.pdf

Unification of XML Documents with Concurrent Markup

In an attempt to put a Semantic Web-layer that provides linguistic analysis and discourse information on top of digital content, we develop a platform for digital curation technologies. The platform offers language-, knowledge- and data-aware services as a flexible set of workflows and pipelines for the efficient processing of various types of digital content. The platform is intended to enable human experts (knowledge workers) to get a grasp and understand the contents of large document collections in an efficient way so that they can curate, process and further analyse the collection according to their sector-specific needs.

Towards a Platform for Curation Technologies: Enriching Text Collections with a Semantic-Web Layer

The technology described in this document “Internationalization Tag Set (ITS) 2.0“ enhances the foundation to integrate automated processing of human language into core Web technologies. ITS 2.0 bears many commonalities with its predecessor, ITS 1.0 but provides additional concepts that are designed to foster the automated creation and processing of multilingual Web content. ITS 2.0 focuses on HTML, XML-based formats in general, and can leverage processing based on the XML Localization Interchange File Format (XLIFF), as well as the Natural Language Processing Interchange Format (NIF). Status of this Document This section describes the status of this document at the time of its publication. Other documents may supersede this document. A list of current W3C publications and the latest revision of this technical report can be found in the W3C technical reports index at http://www.w3.org/TR/. The technology described in this document “Internationalization Tag Set (ITS) 2.0“ enhances the foundation to integrate automated processing of human language into core Web technologies. ITS 2.0 bears many commonalities with is predecessor, ITS 1.0 but provides additional concepts that are designed to foster the automated creation and processing of multilingual Web content. ITS 2.0 focuses on HTML, XML-based formats in general, and can leverage processing based on the XML Localization Interchange File Format (XLIFF), as well as the Natural Language Processing Interchange Format (NIF). This document was published by the MultilingualWeb-LT Working Group as a W3C Recommendation (see W3C document maturity levels). The Working Group has completed and approved this specification's Test Suite and created an Implementation Report that shows that two or more independent implementations pass each test. This document has been reviewed by W3C Members, by software developers, and by other W3C groups and interested parties, and is endorsed by the Director as a W3C Recommendation. It is a stable document and may be used as reference material or cited from another document. W3C's role in making the Recommendation is to draw attention to the specification and to promote its widespread deployment. This enhances the functionality and interoperability of the Web. The ITS 2.0 specification has a normative dependency on the HTML5 specification: it relies on the HTML5 Translate attribute. By publishing this Recommendation, W3C expects that the functionality specified in this ITS 2.0 Recommendation will not be affected by changes to HTML5 as that specification proceeds to Recommendation. If you wish to make comments, please send them to public-i18n-its-ig@w3.org. The archives for this list are publicly available. See also issues discussed within the MultilingualWeb-LT Working Group and the list of changes since the previous publication. This document was produced by a group operating under the 5 February 2004 W3C Patent Policy. W3C maintains a public list of any patent disclosures made in connection with the deliverables of the group; that page also includes instructions for disclosing a patent. An individual who has actual knowledge of a patent which the individual believes contains Essential Claim(s) must disclose the information in accordance with section 6 of the W3C Patent Policy.

Internationalization Tag Set (ITS) Version 2.0

This article introduces the topic of “Multilingual language resources and interoperability”. We start with a taxonomy and parameters for classifying language resources. Later we provide examples and issues of interoperatability, and resource architectures to solve such issues. Finally we discuss aspects of linguistic formalisms and interoperability.

/pdf/multilingual-language-resources-and-interoperability-2gzh3du07n.pdf

Multilingual language resources and interoperability

Corpora annotated for discourse–related phenomena have become an important source for the empirical study of whole texts and a source of training data for the automated parsing of whole texts. By discourse–related phenomena, we refer to various textual relations (e.g., anaphoric and rhetorical relations) that hold among textual units (e.g., whole texts, adjacency pairs, and discourse segments). One of the main problems associated with corpus linguistics—and for the markup of linguistic data in general—has been the lack of interoperability between disparately marked up resources. While it is theoretically possible to standardize the content and structure of markup, the richness of the data itself suggests that no one standard scheme would suffice for all data, all of the time. In fact one of the major contributions of the E–MELD project1 has been to show that annotation elements need not be standardized as such. Rather, by following certain parameters of “best–practice” (Bird & Simons 2003) the stage can be set for wide–spread interoperability among disparate corpora. For example, one of the key strategies of the E–MELD project has been to suggest that all markup elements used in annotating linguistic data (including discourse–related corpora) should be mapped to a semantic resource that defines the meaning of each element. Such a semantic resource for descriptive discourse categories does not exist at present. To rectify this situation, we propose a discourse–specific extension to the General Ontology for Linguistic Description (GOLD), as introduced by Farrar & Langendoen (2003) and explicated in Farrar (forthcoming).

Felix Sasaki

Papers

Unification of XML Documents with Concurrent Markup

Towards a Platform for Curation Technologies: Enriching Text Collections with a Semantic-Web Layer

Internationalization Tag Set (ITS) Version 2.0

Multilingual language resources and interoperability

GOLD and Discourse: Domain- and Community-Specific Extensions