Proceedings ArticleDOI
Friendly Topic Assistant for Transformer Based Abstractive Summarization
Zhengjue Wang,Zhibin Duan,Hao Zhang,Chaojie Wang,Long Tian,Bo Chen,Mingyuan Zhou +6 more
- pp 485-497
Reads0
Chats0
TLDR
A topic assistant (TA) including three modules is proposed that is compatible with various Transformer-based models and user-friendly since i) TA is a plug-and-play model that does not break any structure of the original Transformer network, making users easily fine-tune Transformer+TA based on a well pre-trained model.Abstract:
ive document summarization is a comprehensive task including document understanding and summary generation, in which area Transformer-based models have achieved the state-of-the-art performance. Compared with Transformers, topic models are better at learning explicit document semantics, and hence could be integrated into Transformers to further boost their performance. To this end, we rearrange and explore the semantics learned by a topic model, and then propose a topic assistant (TA) including three modules. TA is compatible with various Transformer-based models and user-friendly since i) TA is a plug-and-play model that does not break any structure of the original Transformer network, making users easily fine-tune Transformer+TA based on a well pre-trained model; ii) TA only introduces a small number of extra parameters. Experimental results on three datasets demonstrate that TA is able to improve the performance of several Transformer-based models.read more
Citations
More filters
Proceedings ArticleDOI
Re-Examining System-Level Correlations of Automatic Summarization Evaluation Metrics
TL;DR: This work identifies two ways in which the definition of the system-level correlation is inconsistent with how metrics are used to evaluate systems in practice and proposes changes to rectify this disconnect.
Posted Content
FFCI: A Framework for Interpretable Automatic Evaluation of Summarization.
TL;DR: This paper constructs a novel dataset for focus, coverage, and inter-sentential coherence, and develops automatic methods for evaluating each of the four dimensions of FFCI based on cross-comparison of evaluation metrics and model-based evaluation methods.
Journal ArticleDOI
Document Summarization with Latent Queries
Yumo Xu,Mirella Lapata +1 more
TL;DR: This framework formulates summarization as a generative process, and jointly optimizes a latent query model and a conditional language model, and outperforms strong comparison systems across benchmarks, query types, document settings, and target domains.
Proceedings ArticleDOI
GRETEL: Graph Contrastive Topic Enhanced Language Model for Long Document Extractive Summarization
TL;DR: A novel model that incorporates the graph contrastive topic model with the pre-trained language model, to fully leverage both the global and local contextual semantics for long document extractive summarization, and outperforms SOTA methods.
References
More filters
Proceedings Article
Attention is All you Need
Ashish Vaswani,Noam Shazeer,Niki Parmar,Jakob Uszkoreit,Llion Jones,Aidan N. Gomez,Lukasz Kaiser,Illia Polosukhin +7 more
TL;DR: This paper proposed a simple network architecture based solely on an attention mechanism, dispensing with recurrence and convolutions entirely and achieved state-of-the-art performance on English-to-French translation.
Journal ArticleDOI
Latent dirichlet allocation
TL;DR: This work proposes a generative model for text and other collections of discrete data that generalizes or improves on several previous models including naive Bayes/unigram, mixture of unigrams, and Hofmann's aspect model.
Proceedings Article
Latent Dirichlet Allocation
TL;DR: This paper proposed a generative model for text and other collections of discrete data that generalizes or improves on several previous models including naive Bayes/unigram, mixture of unigrams, and Hof-mann's aspect model, also known as probabilistic latent semantic indexing (pLSI).
Proceedings ArticleDOI
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
TL;DR: BERT as mentioned in this paper pre-trains deep bidirectional representations from unlabeled text by jointly conditioning on both left and right context in all layers, which can be fine-tuned with just one additional output layer to create state-of-the-art models for a wide range of tasks.
Proceedings Article
ROUGE: A Package for Automatic Evaluation of Summaries
TL;DR: Four different RouGE measures are introduced: ROUGE-N, ROUge-L, R OUGE-W, and ROUAGE-S included in the Rouge summarization evaluation package and their evaluations.