Open AccessProceedings Article
Smatch: an Evaluation Metric for Semantic Feature Structures
Shu Cai,Kevin Knight +1 more
- pp 748-752
Reads0
Chats0
TLDR
This paper presents smatch, a metric that calculates the degree of overlap between two semantic feature structures, and gives an efficient algorithm to compute the metric and shows the results of an inter-annotator agreement study.Abstract:
The evaluation of whole-sentence semantic structures plays an important role in semantic parsing and large-scale semantic structure annotation. However, there is no widely-used metric to evaluate wholesentence semantic structures. In this paper, we present smatch, a metric that calculates the degree of overlap between two semantic feature structures. We give an efficient algorithm to compute the metric and show the results of an inter-annotator agreement study.read more
Citations
More filters
Journal ArticleDOI
Better Smatch = Better Parser? AMR evaluation is not so simple anymore
Juri Opitz,Anette Frank +1 more
TL;DR: An analysis of two popular and strong AMR parsers that reach quality levels on par with human IAA, and assess how human quality rat-ings relate to S MATCH and other AMR metrics.
Posted Content
A Type-coherent, Expressive Representation as an Initial Step to Language Understanding
TL;DR: In this paper, an initial form for a semantic representation for Episodic Logic (EL) is described, and a divide-and-conquer approach to semantic parsing starting with derivation of ULFs is proposed.
Proceedings ArticleDOI
Parsing Indonesian Sentence into Abstract Meaning Representation using Machine Learning Approach
TL;DR: This article used dependency parsing component to get the edges between the words for the AMR and then passed the result of pair prediction to the label prediction process which used a supervised learning algorithm to predict the label between the edges of AMR.
Journal ArticleDOI
SBERT studies Meaning Representations: Decomposing Sentence Embeddings into Explainable AMR Meaning Features
Juri Opitz,Anette Frank +1 more
TL;DR: This work creates similarity metrics that are highly effective, while also providing an interpretable rationale for their rating, and employs these metrics to induce Semantically Structured Sentence BERT embeddings (S 3 BERT), which are composed of different meaning aspects captured in different sub-spaces.
Proceedings ArticleDOI
Sequence-to-sequence AMR Parsing with Ancestor Information
Chenyao Yu,Daniel Gildea +1 more
TL;DR: This paper designs several strategies to add the important ancestor information into the Transformer Decoder and shows that they can improve the performance for both AMR 2.0 and AMR 3.0 dataset and achieve new state-of-the-art results.
References
More filters
Proceedings ArticleDOI
Bleu: a Method for Automatic Evaluation of Machine Translation
TL;DR: This paper proposed a method of automatic machine translation evaluation that is quick, inexpensive, and language-independent, that correlates highly with human evaluation, and that has little marginal cost per run.
Proceedings Article
A Study of Translation Edit Rate with Targeted Human Annotation
TL;DR: A new, intuitive measure for evaluating machine translation output that avoids the knowledge intensiveness of more meaning-based approaches, and the labor-intensiveness of human judgments is defined.
Book
Events in the Semantics of English: A Study in Subatomic Semantics
TL;DR: Focusing on the structure of meaning in English sentences at a "subatomic" level - that is, a level below the one most theories accept as basic or "atomic" - Parsons asserts that the semantics of simple English sentences require logical forms somewhat more complex than is normally assumed in natural language semantics.
Proceedings Article
Learning to map sentences to logical form: structured classification with probabilistic categorial grammars
Luke Zettlemoyer,Michael Collins +1 more
TL;DR: A learning algorithm is described that takes as input a training set of sentences labeled with expressions in the lambda calculus and induces a grammar for the problem, along with a log-linear model that represents a distribution over syntactic and semantic analyses conditioned on the input sentence.
Proceedings Article
From treebank to propbank
Paul R. Kingsbury,Martha Palmer +1 more
TL;DR: This paper describes the approach to the development of a Proposition Bank, which involves the addition of semantic information to the Penn English Treebank and introduces metaframes as a technique for handling similar frames among near− synonymous verbs.