Open AccessProceedings Article
Smatch: an Evaluation Metric for Semantic Feature Structures
Shu Cai,Kevin Knight +1 more
- pp 748-752
Reads0
Chats0
TLDR
This paper presents smatch, a metric that calculates the degree of overlap between two semantic feature structures, and gives an efficient algorithm to compute the metric and shows the results of an inter-annotator agreement study.Abstract:
The evaluation of whole-sentence semantic structures plays an important role in semantic parsing and large-scale semantic structure annotation. However, there is no widely-used metric to evaluate wholesentence semantic structures. In this paper, we present smatch, a metric that calculates the degree of overlap between two semantic feature structures. We give an efficient algorithm to compute the metric and show the results of an inter-annotator agreement study.read more
Citations
More filters
Proceedings ArticleDOI
DynamicPower at SemEval-2016 Task 8: Processing syntactic parse trees with a Dynamic Semantics core
TL;DR: This is a system description paper for a submission to Task 8 of SemEval-2016: Meaning Representation Parsing, where existing components were combined to form a pipeline able to take raw sentences as input and output meaning representations.
Book ChapterDOI
Semantic Analysis and Evaluation of Translation Based on Abstract Meaning Representation
TL;DR: In order to explore the semantic equivalence among homologous translations, the paper analyzes the scenarios of same semantic structures covered by AMR and proposes a framework of variation in homologueous translations.
Proceedings Article
Evaluation Strategies for Computational Construction Grammars
TL;DR: A number of evaluation measures are presented, partially based on existing measures in the field of semantic parsing, that are especially relevant for reversible grammar formalisms that are tested on a grammar fragment for European Portuguese clitic placement.
Journal ArticleDOI
Spanish Abstract Meaning Representation: Annotation of a General Corpus
TL;DR: This work presents the first sizable, general annotation project for Spanish Abstract Meaning Representation, which makes use of Spanish rolesets from the AnCora-Net lexicon and extends English AMR with semantic features specific to Spanish.
Proceedings ArticleDOI
DRS Parsing as Sequence Labeling
Minxing Shen,Kilian Evang +1 more
TL;DR: This work presents the first fully trainable semantic parser for English, German, Italian, and Dutch discourse representation structures (DRSs) that is competitive in accuracy with recent sequence-to-sequence models and at the same time "compositional" in the sense that the output maps each token to one of a finite set of meaning.
References
More filters
Proceedings ArticleDOI
Bleu: a Method for Automatic Evaluation of Machine Translation
TL;DR: This paper proposed a method of automatic machine translation evaluation that is quick, inexpensive, and language-independent, that correlates highly with human evaluation, and that has little marginal cost per run.
Proceedings Article
A Study of Translation Edit Rate with Targeted Human Annotation
TL;DR: A new, intuitive measure for evaluating machine translation output that avoids the knowledge intensiveness of more meaning-based approaches, and the labor-intensiveness of human judgments is defined.
Book
Events in the Semantics of English: A Study in Subatomic Semantics
TL;DR: Focusing on the structure of meaning in English sentences at a "subatomic" level - that is, a level below the one most theories accept as basic or "atomic" - Parsons asserts that the semantics of simple English sentences require logical forms somewhat more complex than is normally assumed in natural language semantics.
Proceedings Article
Learning to map sentences to logical form: structured classification with probabilistic categorial grammars
Luke Zettlemoyer,Michael Collins +1 more
TL;DR: A learning algorithm is described that takes as input a training set of sentences labeled with expressions in the lambda calculus and induces a grammar for the problem, along with a log-linear model that represents a distribution over syntactic and semantic analyses conditioned on the input sentence.
Proceedings Article
From treebank to propbank
Paul R. Kingsbury,Martha Palmer +1 more
TL;DR: This paper describes the approach to the development of a Proposition Bank, which involves the addition of semantic information to the Penn English Treebank and introduces metaframes as a technique for handling similar frames among near− synonymous verbs.