scispace - formally typeset
Search or ask a question
Topic

Natural language generation

About: Natural language generation is a research topic. Over the lifetime, 2421 publications have been published within this topic receiving 46555 citations. The topic is also known as: text generation & NLG.


Papers
More filters
Book
28 Jan 2000
TL;DR: The architecture of a Natural Language Generation system and its implications for national language generation in practice are described.
Abstract: This book explains how to build Natural Language Generation (NLG) systems - computer software systems which use techniques from artificial intelligence and computational linguistics to automatically generate understandable texts in English or other human languages, either in isolation or as part of multimedia documents, Web pages, and speech output systems Typically starting from some non-linguistic representation of information as input, NLG systems use knowledge about language and the application domain to automatically produce documents, reports, explanations, help messages, and other kinds of texts The book covers the algorithms and representations needed to perform the core tasks of document planning, microplanning, and surface realization, using a case study to show how these components fit together It also discusses engineering issues such as system architecture, requirements analysis, and the integration of text generation into multimedia and speech output systems

1,600 citations

Proceedings Article
08 May 2019
TL;DR: UniLM as mentioned in this paper is a unified pre-trained language model that can be fine-tuned for both natural language understanding and generation tasks, achieving state-of-the-art results on five natural language generation datasets, including improving the CNN/DailyMail abstractive summarization ROUGE-L to 40.51 (2.04 absolute improvement).
Abstract: This paper presents a new Unified pre-trained Language Model (UniLM) that can be fine-tuned for both natural language understanding and generation tasks. The model is pre-trained using three types of language modeling tasks: unidirectional, bidirectional, and sequence-to-sequence prediction. The unified modeling is achieved by employing a shared Transformer network and utilizing specific self-attention masks to control what context the prediction conditions on. UniLM compares favorably with BERT on the GLUE benchmark, and the SQuAD 2.0 and CoQA question answering tasks. Moreover, UniLM achieves new state-of-the-art results on five natural language generation datasets, including improving the CNN/DailyMail abstractive summarization ROUGE-L to 40.51 (2.04 absolute improvement), the Gigaword abstractive summarization ROUGE-L to 35.75 (0.86 absolute improvement), the CoQA generative question answering F1 score to 82.5 (37.1 absolute improvement), the SQuAD question generation BLEU-4 to 22.12 (3.75 absolute improvement), and the DSTC7 document-grounded dialog response generation NIST-4 to 2.67 (human performance is 2.65). The code and pre-trained models are available at https://github.com/microsoft/unilm.

1,019 citations

Journal ArticleDOI
TL;DR: The authors examined the problem of generating definite noun phrases that are appropriate referring expressions, that is, noun phrases which successfully identify the intended referent to the hearer whilst not conveying to him or her any false conversational implicatures.

759 citations

Proceedings ArticleDOI
07 Aug 2015
TL;DR: A statistical language generator based on a semantically controlled Long Short-term Memory (LSTM) structure that can learn from unaligned data by jointly optimising sentence planning and surface realisation using a simple cross entropy training criterion, and language variation can be easily achieved by sampling from output candidates.
Abstract: Natural language generation (NLG) is a critical component of spoken dialogue and it has a significant impact both on usability and perceived quality. Most NLG systems in common use employ rules and heuristics and tend to generate rigid and stylised responses without the natural variation of human language. They are also not easily scaled to systems covering multiple domains and languages. This paper presents a statistical language generator based on a semantically controlled Long Short-term Memory (LSTM) structure. The LSTM generator can learn from unaligned data by jointly optimising sentence planning and surface realisation using a simple cross entropy training criterion, and language variation can be easily achieved by sampling from output candidates. With fewer heuristics, an objective evaluation in two differing test domains showed the proposed method improved performance compared to previous methods. Human judges scored the LSTM system higher on informativeness and naturalness and overall preferred it to the other systems..

726 citations

Journal ArticleDOI
TL;DR: This article used the boosting approach to rerank the output of an existing probabilistic parser using additional features of the tree as evidence, without concerns about how these features interact or overlap and without the need to define a derivation or a generative model which takes these features into account.
Abstract: This article considers approaches which rerank the output of an existing probabilistic parser. The base parser produces a set of candidate parses for each input sentence, with associated probabilities that define an initial ranking of these parses. A second model then attempts to improve upon this initial ranking, using additional features of the tree as evidence. The strength of our approach is that it allows a tree to be represented as an arbitrary set of features, without concerns about how these features interact or overlap and without the need to define a derivation or a generative model which takes these features into account. We introduce a new method for the reranking task, based on the boosting approach to ranking problems described in Freund et al. (1998). We apply the boosting method to parsing the Wall Street Journal treebank. The method combined the log-likelihood under a baseline model (that of Collins [1999]) with evidence from an additional 500,000 features over parse trees that were not included in the original model. The new model achieved 89.75% F-measure, a 13% relative decrease in F measure error over the baseline model's score of 88.2%. The article also introduces a new algorithm for the boosting approach which takes advantage of the sparsity of the feature space in the parsing data. Experiments show significant efficiency gains for the new algorithm over the obvious implementation of the boosting approach. We argue that the method is an appealing alternative—in terms of both simplicity and efficiency—to work on feature selection methods within log-linear (maximum-entropy) models. Although the experiments in this article are on natural language parsing (NLP), the approach should be applicable to many other NLP problems which are naturally framed as ranking tasks, for example, speech recognition, machine translation, or natural language generation.

725 citations


Network Information
Related Topics (5)
Natural language
31.1K papers, 806.8K citations
87% related
Recommender system
27.2K papers, 598K citations
83% related
Supervised learning
20.8K papers, 710.5K citations
81% related
Feature learning
15.5K papers, 684.7K citations
80% related
Semi-supervised learning
12.1K papers, 611.2K citations
80% related
Performance
Metrics
No. of papers in the topic in previous years
YearPapers
202381
2022163
2021241
2020268
2019232
2018154