Topic detection is a challenging task, especially without knowing the exact number of topics In this paper, we present a novel approach based on neural network to detect topics in the micro-blogging dataset We use an unsupervised neural sentence embedding model to map the blogs to an embedding space Our model is a weighted power mean word embedding model, and the weights are calculated by attention mechanism Experimental result shows our embedding method performs better than baselines in sentence clustering In addition, we propose an improved clustering algorithm referred as relationship-aware DBSCAN (RADBSCAN) It can discover topics from a micro-blogging dataset, and the topic number depends on dataset character itself Moreover, in order to solve the problem of parameters sensitive, we take blog forwarding relationship as a bridge of two independent clusters Finally, we validate our approach on a dataset from sina micro-blog The result shows that we can detect all the topics successfully and extract keywords in each topic

A novel sentence embedding based topic detection method for micro-blog

Topic detection is a difficult challenging task, especially when the exact number of topics is unknown. In this article, we present a novel topic detection approach based on neural computing to detect topics in a microblogging dataset. We use an unsupervised neural sentence embedding model to map blogs to an embedding space. The proposed model is a weighted power mean sentence embedding model in which weights are calculated by a targeted attention mechanism. The experimental results show that our embedding model performs better than baseline in sentence clustering. In addition, we propose a clustering algorithm, referred to as Relationship-Aware DBSCAN (RADBSCAN), to discover topics from a microblogging dataset in which the number of topics is automatically determined by the characteristics of the dataset. Moreover, to provide parameter insensibility, we use the forwarding relationship in the blogs as a bridge of two independent clusters. Finally, we validate the proposed method on a dataset from the Sina microblog. The results show that our approach can detect all topics successfully and can extract the keywords of each topic.

A Novel Sentence Embedding Based Topic Detection Method for Microblogs

Polyphone disambiguation aims to capture accurate pronunciation knowledge from natural text sequences for reliable Text-to-speech (TTS) systems. However, previous approaches require substantial annotated training data and additional efforts from language experts, making it difficult to extend high-quality neural TTS systems to out-of-domain daily conversations and countless languages worldwide. This paper tackles the polyphone disambiguation problem from a concise and novel perspective: we propose Dict-TTS, a semantic-aware generative text-to-speech model with an online website dictionary (the existing prior information in the natural language). Specifically, we design a semantics-to-pronunciation attention (S2PA) module to match the semantic patterns between the input text sequence and the prior semantics in the dictionary and obtain the corresponding pronunciations; The S2PA module can be easily trained with the end-to-end TTS model without any annotated phoneme labels. Experimental results in three languages show that our model outperforms several strong baseline models in terms of pronunciation accuracy and improves the prosody modeling of TTS systems. Further extensive analyses demonstrate that each design in Dict-TTS is effective. The code is available at \url{https://github.com/Zain-Jiang/Dict-TTS}.

Dict-TTS: Learning to Pronounce with Prior Dictionary Knowledge for Text-to-Speech

Large pre-trained language models (LPLM) have shown spectacular success when ﬁne-tuned on downstream supervised tasks. Yet, it is known that their performance can drastically drop when there is a distribution shift between the data used during training and that used at inference time. In this paper we focus on data distributions that naturally change over time and introduce four new R EDDIT datasets, namely the W ALLSTREETBETS , A SK S CIENCE , T HE D ONALD , and P OLITICS sub-reddits. First, we empirically demonstrate that LPLM can display average performance drops of about 88 % (in the best case!) when predicting the popularity of future posts from sub-reddits whose topic distribution changes with time . We then introduce a simple methodology that leverages neural variational dynamic topic models and attention mechanisms to infer temporal language model representations for regression tasks. Our models display performance drops of only about 40 % in the worst cases (2 % in the best ones) when predicting the popularity of future posts, while using only about 7 % of the total number of parameters of LPLM and providing interpretable representations that offer insight into real-world events, like the GameStop short squeeze of 2021.

The future is different: Large pre-trained language models fail in prediction tasks

Abstract Under the influence of the health emergency triggered by the COVID-19 pandemic, many brands changed their communication strategy and included more or less explicit references to the principles of solidarity and fraternity in their TV commercials to boost the confidence and hope of Italian families during the lockdown. The traditional attitudes of the advertising format, which focused on product characteristics, were relegated to the background in order to reinforce the “brand image” through words, signs, hashtags and music that spread empathetic messages to all those who needed to regain hope and trust in a time of extreme emotional fragility. The objective of this paper is to identify the emotions and brand awareness during the lockdown using text mining techniques by measuring customer sentiment expressed on the Twitter social network. Our proposal starts from an unstructured corpus of 20,982 tweets processed with text data mining techniques to identify patterns and trends in people’s posts related to specific hashtags and TV ads produced during the COVID-19 pandemic. The innovations in the brand’s advertising among consumers seem to have triggered some sense of appreciation and gratitude, as well as a strong sense of belonging that was not present before, as the TV ads were perceived as a disruptive element in consumers’ tweets. Although this effect is clearly documented, in this paper we demonstrate its transitory nature, in the sense that the frequency of occurrence of terms associated with an emotional dimension peaks during the weeks of lockdown, and then gradually decreases. 

https://link.springer.com/content/pdf/10.1007/s11135-022-01460-3.pdf

A text data mining approach to the study of emotions triggered by new advertising formats during the COVID-19 pandemic

For extracting meaningful topics from texts, their structures should be considered properly. In this paper, we aim to analyze structured time-series documents such as a collection of news articles and a series of scientific papers, wherein topics evolve along time depending on multiple topics in the past and are also related to each other at each time. To this end, we propose a dynamic and static topic model, which simultaneously considers the dynamic structures of the temporal topic evolution and the static structures of the topic hierarchy at each time. We show the results of experiments on collections of scientific papers, in which the proposed method outperformed conventional models. Moreover, we show an example of extracted topic structures, which we found helpful for analyzing research activities.

Dynamic and Static Topic Model for Analyzing Time-Series Document Collections

Although end-to-end text-to-speech (TTS) models can generate natural speech, challenges still remain when it comes to estimating sentence-level phonetic and prosodic information from raw text in Japanese TTS systems. In this paper, we propose a method for polyphone disambiguation (PD) and accent prediction (AP). The proposed method incorporates explicit features extracted from morphological analysis and implicit features extracted from pre-trained language models (PLMs). We use BERT and Flair embeddings as implicit features and examine how to combine them with explicit features. Our objective evaluation results showed that the proposed method improved the accuracy by 5.7 points in PD and 6.0 points in AP. Moreover, the perceptual listening test results confirmed that a TTS system employing our proposed model as a front-end achieved a mean opinion score close to that of synthesized speech with ground-truth pronunciation and accent in terms of naturalness.

/pdf/polyphone-disambiguation-and-accent-prediction-using-pre-3k3y9821.pdf

Polyphone Disambiguation and Accent Prediction Using Pre-Trained Language Models in Japanese TTS Front-End

/pdf/dynamic-and-static-topic-model-for-analyzing-time-series-1tf3h8rlfr.pdf

Rem Hida

Papers

Dynamic and Static Topic Model for Analyzing Time-Series Document Collections

Polyphone Disambiguation and Accent Prediction Using Pre-Trained Language Models in Japanese TTS Front-End

Dynamic and Static Topic Model for Analyzing Time-Series Document Collections.