Open AccessProceedings Article
Smatch: an Evaluation Metric for Semantic Feature Structures
Shu Cai,Kevin Knight +1 more
- pp 748-752
Reads0
Chats0
TLDR
This paper presents smatch, a metric that calculates the degree of overlap between two semantic feature structures, and gives an efficient algorithm to compute the metric and shows the results of an inter-annotator agreement study.Abstract:
The evaluation of whole-sentence semantic structures plays an important role in semantic parsing and large-scale semantic structure annotation. However, there is no widely-used metric to evaluate wholesentence semantic structures. In this paper, we present smatch, a metric that calculates the degree of overlap between two semantic feature structures. We give an efficient algorithm to compute the metric and show the results of an inter-annotator agreement study.read more
Citations
More filters
An in-depth look at Euclidean disk embeddings for structure preserving parsing
TL;DR: In this article, the applicability of disk embeddings for embedding Directed Acyclic Graphs (DAGs) has been tested for two structure generation tasks, namely dependency and semantic parsing.
Proceedings ArticleDOI
Making Better Use of Bilingual Information for Cross-Lingual AMR Parsing
Yitao Cai,Zhe Lin,Xiaojun Wan +2 more
TL;DR: This paper introduced bilingual input, namely translated texts as well as non-English texts, in order to enable the model to predict more accurate concepts, and also introduced an auxiliary task, requiring the decoder to predict the English sequences at the same time.
Referenceless Parsing-Based Evaluation of AMR-to-English Generation
Emma Manning,Nathan Schneider +1 more
Proceedings Article
Multilingual AMR Parsing with Noisy Knowledge Distillation
TL;DR: The authors study multilingual AMR parsing from the perspective of knowledge distillation, where the aim is to learn and improve a multilingual parser by using an existing English parser as its teacher.
SMARAGD: Learning SMatch for Accurate and Rapid Approximate Graph Distance
TL;DR: This article proposed SMARAGD (Semantic Matching for Accurate and Rapid Approximate Graph Distance), which is a semantic matching algorithm for graph similarity analysis that uses machine translation and Siamese CNNs.
References
More filters
Proceedings ArticleDOI
Bleu: a Method for Automatic Evaluation of Machine Translation
TL;DR: This paper proposed a method of automatic machine translation evaluation that is quick, inexpensive, and language-independent, that correlates highly with human evaluation, and that has little marginal cost per run.
Proceedings Article
A Study of Translation Edit Rate with Targeted Human Annotation
TL;DR: A new, intuitive measure for evaluating machine translation output that avoids the knowledge intensiveness of more meaning-based approaches, and the labor-intensiveness of human judgments is defined.
Book
Events in the Semantics of English: A Study in Subatomic Semantics
TL;DR: Focusing on the structure of meaning in English sentences at a "subatomic" level - that is, a level below the one most theories accept as basic or "atomic" - Parsons asserts that the semantics of simple English sentences require logical forms somewhat more complex than is normally assumed in natural language semantics.
Proceedings Article
Learning to map sentences to logical form: structured classification with probabilistic categorial grammars
Luke Zettlemoyer,Michael Collins +1 more
TL;DR: A learning algorithm is described that takes as input a training set of sentences labeled with expressions in the lambda calculus and induces a grammar for the problem, along with a log-linear model that represents a distribution over syntactic and semantic analyses conditioned on the input sentence.
Proceedings Article
From treebank to propbank
Paul R. Kingsbury,Martha Palmer +1 more
TL;DR: This paper describes the approach to the development of a Proposition Bank, which involves the addition of semantic information to the Penn English Treebank and introduces metaframes as a technique for handling similar frames among near− synonymous verbs.