The task of automatic document summarization aims at generating short summaries for originally long documents. A good summary should cover the most important information of the original document or a cluster of documents, while being coherent, non-redundant and grammatically readable. Numerous approaches for automatic summarization have been developed to date. In this paper we give a self-contained, broad overview of recent progress made for document summarization within the last 5 years. Specifically, we emphasize on significant contributions made in recent years that represent the state-of-the-art of document summarization, including progress on modern sentence extraction approaches that improve concept coverage, information diversity and content coherence, as well as attempts from summarization frameworks that integrate sentence compression, and more abstractive systems that are able to produce completely new sentences. In addition, we review progress made for document summarization in domains, genres and applications that are different from traditional settings. We also point out some of the latest trends and highlight a few possible future directions.

Recent advances in document summarization

In existing unsupervised methods, Latent Semantic Analysis (LSA) is used for sentence selection. However, the obtained results are less meaningful, because singular vectors are used as the bases for sentence selection from given documents, and singular vector components can have negative values. We propose a new unsupervised method using Non-negative Matrix Factorization (NMF) to select sentences for automatic generic document summarization. The proposed method uses non-negative constraints, which are more similar to the human cognition process. As a result, the method selects more meaningful sentences for generic document summarization than those selected using LSA.

Automatic generic document summarization based on non-negative matrix factorization ; Automatic generic document summarization based on non-negative matrix factorization

Information Retrieval.- A New Method for Sentiment Classification in Text Retrieval.- Topic Tracking Based on Linguistic Features.- The Use of Monolingual Context Vectors for Missing Translations in Cross-Language Information Retrieval.- Automatic Image Annotation Using Maximum Entropy Model.- Corpus-Based Parsing.- Corpus-Based Analysis of Japanese Relative Clause Constructions.- Parsing Biomedical Literature.- Parsing the Penn Chinese Treebank with Semantic Knowledge.- Using a Partially Annotated Corpus to Build a Dependency Parser for Japanese.- Web Mining.- Entropy as an Indicator of Context Boundaries: An Experiment Using a Web Search Engine.- Automatic Discovery of Attribute Words from Web Documents.- Aligning Needles in a Haystack: Paraphrase Acquisition Across the Web.- Confirmed Knowledge Acquisition Using Mails Posted to a Mailing List.- Rule-Based Parsing.- Automatic Partial Parsing Rule Acquisition Using Decision Tree Induction.- Chunking Using Conditional Random Fields in Korean Texts.- High Efficiency Realization for a Wide-Coverage Unification Grammar.- Linguistically-Motivated Grammar Extraction, Generalization and Adaptation.- Disambiguation.- PP-Attachment Disambiguation Boosted by a Gigantic Volume of Unambiguous Examples.- Adapting a Probabilistic Disambiguation Model of an HPSG Parser to a New Domain.- A Hybrid Approach to Single and Multiple PP Attachment Using WordNet.- Period Disambiguation with Maxent Model.- Text Mining.- Acquiring Synonyms from Monolingual Comparable Texts.- A Method of Recognizing Entity and Relation.- Inversion Transduction Grammar Constraints for Mining Parallel Sentences from Quasi-Comparable Corpora.- Automatic Term Extraction Based on Perplexity of Compound Words.- Document Analysis.- Document Clustering with Grouping and Chaining Algorithms.- Using Multiple Discriminant Analysis Approach for Linear Text Segmentation.- Classifying Chinese Texts in Two Steps.- Assigning Polarity Scores to Reviews Using Machine Learning Techniques.- Ontology and Thesaurus.- Analogy as Functional Recategorization: Abstraction with HowNet Semantics.- PLSI Utilization for Automatic Thesaurus Construction.- Analysis of an Iterative Algorithm for Term-Based Ontology Alignment.- Finding Taxonomical Relation from an MRD for Thesaurus Extension.- Relation Extraction.- Relation Extraction Using Support Vector Machine.- Discovering Relations Between Named Entities from a Large Raw Corpus Using Tree Similarity-Based Clustering.- Automatic Relation Extraction with Model Order Selection and Discriminative Label Identification.- Mining Inter-Entity Semantic Relations Using Improved Transductive Learning.- Text Classification.- A Preliminary Work on Classifying Time Granularities of Temporal Questions.- Classification of Multiple-Sentence Questions.- Transliteration.- A Rule Based Syllabification Algorithm for Sinhala.- An Ensemble of Grapheme and Phoneme for Machine Transliteration.- Machine Translation - I.- Improving Statistical Word Alignment with Ensemble Methods.- Empirical Study of Utilizing Morph-Syntactic Information in SMT.- Question Answering.- Instance-Based Generation for Interactive Restricted Domain Question Answering Systems.- Answering Definition Questions Using Web Knowledge Bases.- Exploring Syntactic Relation Patterns for Question Answering.- Web-Based Unsupervised Learning for Query Formulation in Question Answering.- Morphological Analysis.- A Chunking Strategy Towards Unknown Word Detection in Chinese Word Segmentation.- A Lexicon-Constrained Character Model for Chinese Morphological Analysis.- Relative Compositionality of Multi-word Expressions: A Study of Verb-Noun (V-N) Collocations.- Automatic Extraction of Fixed Multiword Expressions.- Machine Translation - II.- Phrase-Based Statistical Machine Translation: A Level of Detail Approach.- Why Is Zero Marking Important in Korean?.- A Phrase-Based Context-Dependent Joint Probability Model for Named Entity Translation.- Machine Translation Based on Constraint-Based Synchronous Grammar.- Text Summarization.- A Machine Learning Approach to Sentence Ordering for Multidocument Summarization and Its Evaluation.- Significant Sentence Extraction by Euclidean Distance Based on Singular Value Decomposition.- Named Entity Recognition.- Two-Phase Biomedical Named Entity Recognition Using A Hybrid Method.- Heuristic Methods for Reducing Errors of Geographic Named Entities Learned by Bootstrapping.- Linguistic Resources and Tools.- Building a Japanese-Chinese Dictionary Using Kanji/Hanzi Conversion.- Automatic Acquisition of Basic Katakana Lexicon from a Given Corpus.- CTEMP: A Chinese Temporal Parser for Extracting and Normalizing Temporal Information.- French-English Terminology Extraction from Comparable Corpora.- Discourse Analysis.- A Twin-Candidate Model of Coreference Resolution with Non-Anaphor Identification Capability.- Improving Korean Speech Acts Analysis by Using Shrinkage and Discourse Stack.- Anaphora Resolution for Biomedical Literature by Exploiting Multiple Resources.- Automatic Slide Generation Based on Discourse Structure Analysis.- Semantic Analysis - I.- Using the Structure of a Conceptual Network in Computing Semantic Relatedness.- Semantic Role Labelling of Prepositional Phrases.- Global Path-Based Refinement of Noisy Graphs Applied to Verb Semantics.- Semantic Role Tagging for Chinese at the Lexical Level.- NLP Applications.- Detecting Article Errors Based on the Mass Count Distinction.- Principles of Non-stationary Hidden Markov Model and Its Applications to Sequence Labeling Task.- Integrating Punctuation Rules and Naive Bayesian Model for Chinese Creation Title Recognition.- A Connectionist Model of Anticipation in Visual Worlds.- Tagging.- Automatically Inducing a Part-of-Speech Tagger by Projecting from Multiple Source Languages Across Aligned Corpora.- The Verbal Entries and Their Description in a Grammatical Information-Dictionary of Contemporary Tibetan.- Tense Tagging for Verbs in Cross-Lingual Context: A Case Study.- Regularisation Techniques for Conditional Random Fields: Parameterised Versus Parameter-Free.- Semantic Analysis - II.- Exploiting Lexical Conceptual Structure for Paraphrase Generation.- Word Sense Disambiguation by Relative Selection.- Towards Robust High Performance Word Sense Disambiguation of English Verbs Using Rich Linguistic Features.- Automatic Interpretation of Noun Compounds Using WordNet Similarity.- Language Models.- An Empirical Study on Language Model Adaptation Using a Metric of Domain Similarity.- A Comparative Study of Language Models for Book and Author Recognition.- Spoken Language.- Lexical Choice via Topic Adaptation for Paraphrasing Written Language to Spoken Language.- A Case-Based Reasoning Approach for Speech Corpus Generation.- Terminology Mining.- Web-Based Terminology Translation Mining.- Extracting Terminologically Relevant Collocations in the Translation of Chinese Monograph.

Natural language processing : IJCNLP 2005 : Second International Joint Conference, Jeju Island, Korea, October 11-13, 2005 : proceedings

In this paper, we propose an extractive multi-document summarization (MDS) system using joint optimization and active learning for content selection grounded in user feedback. Our method interactively obtains user feedback to gradually improve the results of a state-of-the-art integer linear programming (ILP) framework for MDS. Our methods complement fully automatic methods in producing high-quality summaries with a minimum number of iterations and feedbacks. We conduct multiple simulation-based experiments and analyze the effect of feedback-based concept selection in the ILP setup in order to maximize the user-desired content in the summary.

/pdf/joint-optimization-of-user-desired-content-in-multi-document-4evy07ztx2.pdf

Joint Optimization of User-desired Content in Multi-document Summaries by Learning from User Feedback

Many state-of-the-art citation recommendation methods have been proposed for finding a list of reference papers for a given manuscript, among which the graph-based method has gained particular attention, due to its flexibility for incorporating various information that embodies user’s preferences. To achieve a more synthetic, accurate, and personalized recommendation result than the previous graph-based methods, this paper proposes a new graph-based recommendation framework that exploiting diversified link information in a bibliographic network and the concise query information that embodies the specific requirement of user comprehensively. The proposed framework not only performs mutual reinforcement rules on all available multiple types of relations in a multi-layered graph but also incorporates the query information into the multi-layered mutual reinforcement schema to construct a multi-layered mutually reinforced query-focused (MMRQ) citation recommendation approach. Extensive experiments have been conducted on a subset of anthology network data set. Experimental results of Recall measures, normalized discounted cumulative gain measures, and case study all demonstrate that our MMRQ method obtains a superior citation recommendation.

Query-Focused Personalized Citation Recommendation With Mutually Reinforced Ranking

Heavy research has been done in recent years on tasks of traditional summarization. However, social context, which is critical in building high-quality social summarizer for web documents, is usually neglected. To address this issue, we propose a novel summarization approach based on social context. In this approach, social summarization is implemented by first employing the tripartite clustering algorithm to simultaneously discover document context and user context for a specified document. Then sentence relationships intra and inter documents plus intended user communities are taken into account to evaluate the significance of each sentence in different context views. Finally, a few sentences with highest overall scores are selected to form the summary. Experimental results demonstrate the effectiveness of the proposed approach and show the superior performance over several baselines.

/pdf/social-summarization-via-automatically-discovered-social-25yd51orqw.pdf

Social Summarization via Automatically Discovered Social Context

This work investigates an interesting and challenging task in summarization, i.e., personalized social summarization, which aims to adapt summarization result of a specified document to an intended user based on his interests inferred from social context implicitly. Most existing summarization systems generate a uniform version of summary for different users no matter who is reading or generate personalized summaries employing only the local information in the document and the user profile. This paper proposes a novel unsupervised approach by making use of enhanced social context to aid personalized summary generation. In the proposed method, document expansion, user expansion, and implicit induction of the intended user's interest aspects are achieved simultaneously by adopting a fuzzy tripartite clustering algorithm. And both the informativeness of sentences and the user’s interest aspects are incorporated in a unified ranking process. Preliminary experimental results on a social tagging dataset validate the effectiveness of the proposed approach.

Context-Enhanced Personalized Social Summarization

Graph-based ranking methods have been successfully applied to multi-document summarization by adopting various link analysis algorithms such as PageRank and HITS to incorporate diverse relationships into the process of sentence evaluation. Both the homogeneous relationships between sentences and the heterogeneous relationships between sentences and documents have been investigated in the past. However, for query-focused multi-document summarization, the other three kinds of relationships (i.e. the relationships between documents, the relationships between the given query and documents, and the sentence-to-document correlation strength) are seldom considered when computing the sentence’s importance. In order to address the limitations, this study proposes a novel Co-HITS-Ranking based approach to query-biased summarization, which can fuse all of the above relationships, either homogeneous or heterogeneous, in a unified two-layer graph model with the assumption that significant sentences and significant documents can be self boosted and mutually boosted. In the model, the manifold-ranking algorithm is employed to assign the initial biased information richness scores for sentences and documents individually only based on the local recommendations between homogeneous objects. Then by adopting the Co-HITS-Ranking algorithm, the initial biased information richness scores of sentences and documents are naturally incorporated in a mutual reinforcement framework to co-rank heterogeneous objects jointly. The final score of each sentence can be obtained through an iteratively updating process. Experimental results on the DUC datasets demonstrate the good effectiveness of the proposed approach.

Co-HITS-Ranking Based Query-Focused Multi-document Summarization

Current researches on Question Answering concern more complex questions than factoid ones. Since complex questions are investigated by many researches, how to acquire accurate answers still becomes a core problem for complex QA. In this paper, we propose an approach that estimates the similarity by topic model. After summarizing relevant texts from web knowledge bases, an answer sentence acquisition model based on Probabilistic Latent Semantic Analysis is introduced to seek sentences, in which the topic is similar to those in definition set. Then, an answer ranking model is employed to select both statistically and semantically similar sentences between sentences retrieved and sentences in the relevant text set. Finally, sentences are ranked as answer candidates according to their scores. Experiments show that our approach achieves an increase of 5.19% F-score than the baseline system.

A web knowledge based approach for complex question answering

We propose a collaborative approach to improve document summarization by incorporating social contextual information into the sentence ranking process. Both the relationships between sentences from document context and the preference information from user context are investigated in the approach. We validate our method on a social tagging dataset and experimentally demonstrate that by incorporating social contextual information it obtains significant improvement over several baseline methods.

Chong Teng

Papers

Social Summarization via Automatically Discovered Social Context

Context-Enhanced Personalized Social Summarization

Co-HITS-Ranking Based Query-Focused Multi-document Summarization

A web knowledge based approach for complex question answering

Improving document summarization by incorporating social contextual information