The past several years have witnessed the rapid development of social media services, and the UGCs User Generated Contents have been increased dramatically, such as tweets in Twitter and posts in Sina Weibo. In this paper, we describe our system at NLPCC2015 on the Weibo-oriented Chinese news summarization task. Our model is established based on multi-feature combination to automatically generate summary for the given news article. In our system, we mainly utilize four kinds of features to compute the significance score of a sentence, including term frequency, sentence position, sentence length and the similarity between sentence and news article title, and then the summary sentences are chosen according to the significance score of each sentence from the news article. The evaluation results on Weibo news document sets show that our system is efficient in Weibo-oriented Chinese news summarization and outperforms all the other systems.

Weibo-Oriented Chinese News Summarization via Multi-feature Combination

With worldwide events happening everyday, it’s useful to discover the most discussed topics for intelligence and trend analysis. However, existing news media only provide manually annotated tags related to each article. Given huge amount of Web data with new or unknown terms, it’s critical if these topic-related keywords can be automatically extracted in an efficient way. In this paper, we propose a multi-document term extraction approach to major concept detection. First, major keywords occurring in multiple documents are extracted by statistical term extraction methods. These are then utilized as the dictionary for word segmentation to improve named entity recognition, by which the most relevant concepts in each single document can be extracted. Finally, the extracted concepts are ranked by their importance score based on their popularity and average position in documents. In the experimental results for major online news sources in Chinese, exploiting multi-document term extraction helps to improve the effectiveness of named entity recognition, with the best F1-measure of 0.61, a significant 24% improvement over the baseline. By combining the popularity with average position of terms as the importance score, we able to obtain the best precision of 68% when keeping the top-25% content in documents. Keyword extraction results for selected major events further validated the effectiveness of our proposed approach using topic-related news articles. Further investigation is needed to evaluate the performance of the proposed approach with various data sources.

Exploiting Multi-Document Term Extraction to Improve Named Entity Recognition for Major Concept Detection

Given huge amount of daily news articles, it would be helpful to users if the news reading time can be reduced. In this paper, we focus on single-document summarization for Chinese news articles with statistical methods. First, new vocabularies are collected from news articles, and verified with online translation services. These are included as the auxiliary lexicon. Then, statistical word segmentation is done by calculating the relative frequency of overlapping word n-grams. Finally, the sentence importance is estimated as the weighted sum of n-gram scores, and the top-ranked sentences are selected as the summary. The experimental results showed that generated summaries can be effectively clustered in the same group as the original news articles. A great reduction in storage size can be observed while preserving suitable similarity with the original document. This shows the potential of our proposed approach in news summarization. Further investigation is needed to verify in other document domains.

Jeng-Yuan Yang

Papers

Statistical Single-Document Summarization for Chinese News Articles