Open AccessProceedings Article
Developing Infrastructure for the Evaluation of Single and Multi-document Summarization Systems in a Cross-lingual Environment.
Reads0
Chats0
TLDR
This work describes the development of Language and Evaluation Resources for the evaluation of summaries in English and Chinese and focuses on the resources developed that are made available for the research community.Abstract:
We describe our work on the development of Language and Evaluation Resources for the evaluation of summaries in English and Chinese. The language resources include a parallel corpus of English and Chinese texts which are translations of each other, a set of queries in both languages, clusters of documents relevants to each query, sentence relevance measures for each sentence in the document clusters, and manual multi-document summaries at different compression rates. The evaluation resources consist of metrics for measuring the content of automatic summaries against reference summaries. The framework can be used in the evaluation of extractive, non-extractive, single and multi-document summarization. We focus on the resources developed that are made available for the research community.read more
Citations
More filters
Book ChapterDOI
Automatic Text Summarization: Past, Present and Future
Horacio Saggion,Thierry Poibeau +1 more
TL;DR: This paper gives a short overview of summarization methods and evaluation and the number of interesting summarization topics being proposed in different contexts by end users.
Journal Article
Evaluation Measures for Text Summarization
Josef Steinberger,Karel Ježek +1 more
TL;DR: A new evaluation measure for assessing the quality of a summary that can compare a summary with its full text and if abstracts are not available for a given corpus, using the LSA-based measure is an appropriate choice.
Proceedings ArticleDOI
Examining the consensus between human summaries: initial experiments with factoid analysis
Hans van Halteren,Simone Teufel +1 more
TL;DR: This work presents a new approach to summary evaluation which combines two novel aspects, namely (a) content comparison between gold standard summary and system summary via factoids, a pseudo-semantic representation based on atomic information units which can be robustly marked in text, and (b) use of a gold standard consensus summary.
Proceedings ArticleDOI
Meta-evaluation of summaries in a cross-lingual environment using content-based metrics
TL;DR: A framework for the evaluation of summaries in English and Chinese using similarity measures that can be used to evaluate extractive, non-extractive, single and multi-document summarization is described.
Proceedings ArticleDOI
Robust generic and query-based summarisation
TL;DR: A robust summarisation system developed within the GATE architecture that makes use of robust components for semantic tagging and coreference resolution provided by GATE and combines well established statistical techniques developed for the purpose of text summarisation research is presented.
References
More filters
Book
Nonparametric statistics for the behavioral sciences
TL;DR: This is the revision of the classic text in the field, adding two new chapters and thoroughly updating all others as discussed by the authors, and the original structure is retained, and the book continues to serve as a combined text/reference.
Proceedings ArticleDOI
Bleu: a Method for Automatic Evaluation of Machine Translation
TL;DR: This paper proposed a method of automatic machine translation evaluation that is quick, inexpensive, and language-independent, that correlates highly with human evaluation, and that has little marginal cost per run.
Journal ArticleDOI
WordNet: a lexical database for English
TL;DR: WordNet1 provides a more effective combination of traditional lexicographic information and modern computing, and is an online lexical database designed for use under program control.
Journal ArticleDOI
Non-Parametric Statistics for the Behavioral Sciences.
Alan Stuart,S. Siegel +1 more