Open AccessProceedings Article
Smatch: an Evaluation Metric for Semantic Feature Structures
Shu Cai,Kevin Knight +1 more
- pp 748-752
Reads0
Chats0
TLDR
This paper presents smatch, a metric that calculates the degree of overlap between two semantic feature structures, and gives an efficient algorithm to compute the metric and shows the results of an inter-annotator agreement study.Abstract:
The evaluation of whole-sentence semantic structures plays an important role in semantic parsing and large-scale semantic structure annotation. However, there is no widely-used metric to evaluate wholesentence semantic structures. In this paper, we present smatch, a metric that calculates the degree of overlap between two semantic feature structures. We give an efficient algorithm to compute the metric and show the results of an inter-annotator agreement study.read more
Citations
More filters
Proceedings ArticleDOI
A Method of Semantic Redundant Information Filtering for Abstract Meaning Representation Graph
TL;DR: The experimental results show that the semantic redundancy removal method proposed in this paper has significantly improved the ROUGE and Smatch values compared to other word-aligned eliminate redundant methods.
Automatic Enrichment of Abstract Meaning Representations
TL;DR: The authors developed an automated annotation tool which algorithmically enriches AMR graphs to better represent number, (in)definite articles, quantificational determiners, and intensional arguments.
Journal ArticleDOI
Slide, Constrain, Parse, Repeat: Synchronous SlidingWindows for Document AMR Parsing
TL;DR: The authors proposed a transition-based parser with synchronous sliding windows over source and target for document-level AMR parsing, which leverages source-target alignments and constrains decoding to guarantee synchronicity and consistency across overlapping windows.
Posted Content
A Transition-based Parser for Unscoped Episodic Logical Forms.
TL;DR: The authors used a modified cache transition system to learn a sequence-to-sequence model for predicting the transition action sequence in Episodic Logic:Unscoped Logical Form (EL-ULF).
Proceedings Article
Theoretical and Practical Issues in the Semantic Annotation of Four Indigenous Languages
Jens E. L. Van Gysel,Meagan Vigus,Lukas Denk,Andrew Cowell,Rosa Vallejos,Tim O'Gorman,William Croft +6 more
TL;DR: The authors assesses whether typologically trained non-speakers of an indigenous language can feasibly perform semantic annotation using Uniform Meaning Representations, thus allowing for the creation of computational materials without putting further strain on community resources.
References
More filters
Proceedings ArticleDOI
Bleu: a Method for Automatic Evaluation of Machine Translation
TL;DR: This paper proposed a method of automatic machine translation evaluation that is quick, inexpensive, and language-independent, that correlates highly with human evaluation, and that has little marginal cost per run.
Proceedings Article
A Study of Translation Edit Rate with Targeted Human Annotation
TL;DR: A new, intuitive measure for evaluating machine translation output that avoids the knowledge intensiveness of more meaning-based approaches, and the labor-intensiveness of human judgments is defined.
Book
Events in the Semantics of English: A Study in Subatomic Semantics
TL;DR: Focusing on the structure of meaning in English sentences at a "subatomic" level - that is, a level below the one most theories accept as basic or "atomic" - Parsons asserts that the semantics of simple English sentences require logical forms somewhat more complex than is normally assumed in natural language semantics.
Proceedings Article
Learning to map sentences to logical form: structured classification with probabilistic categorial grammars
Luke Zettlemoyer,Michael Collins +1 more
TL;DR: A learning algorithm is described that takes as input a training set of sentences labeled with expressions in the lambda calculus and induces a grammar for the problem, along with a log-linear model that represents a distribution over syntactic and semantic analyses conditioned on the input sentence.
Proceedings Article
From treebank to propbank
Paul R. Kingsbury,Martha Palmer +1 more
TL;DR: This paper describes the approach to the development of a Proposition Bank, which involves the addition of semantic information to the Penn English Treebank and introduces metaframes as a technique for handling similar frames among near− synonymous verbs.