Showing papers on "Hierarchical Dirichlet process published in 2023"

Short Text Clustering in Continuous Time Using Stacked Dirichlet-Hawkes Process with Inverse Cluster Frequency Prior

Abstract: Traditional models for short text clustering ignore the time information associated with the text documents. However, existing works have shown that temporal characteristics of streaming documents are significant features for clustering. In this paper we propose a stacked Dirichlet-Hawkes process with inverse cluster frequency prior as a simple but effective solution for the task of short text clustering using temporal features in continuous time. Based on the classical formulation of the Dirichlet-Hawkes process, our model provides an elegant, theoretically grounded and interpretable solution while performing at par with recent state of the art models in short text clustering.

Multivariate Powered Dirichlet-Hawkes Process

Abstract: The publication time of a document carries a relevant information about its semantic content. The Dirichlet-Hawkes process has been proposed to jointly model textual information and publication dynamics. This approach has been used with success in several recent works, and extended to tackle specific challenging problems –typically for short texts or entangled publication dynamics. However, the prior in its current form does not allow for complex publication dynamics. In particular, inferred topics are independent from each other –a publication about finance is assumed to have no influence on publications about politics, for instance. In this work, we develop the Multivariate Powered Dirichlet-Hawkes Process (MPDHP), that alleviates this assumption. Publications about various topics can now influence each other. We detail and overcome the technical challenges that arise from considering interacting topics. We conduct a systematic evaluation of MPDHP on a range of synthetic datasets to define its application domain and limitations. Finally, we develop a use case of the MPDHP on Reddit data. At the end of this article, the interested reader will know how and when to use MPDHP, and when not to.

Dirichlet-Survival Process: Scalable Inference of Topic-Dependent Diffusion Networks

Abstract: Information spread on networks can be efficiently modeled by considering three features: documents’ content, time of publication relative to other publications, and position of the spreader in the network. Most previous works model up to two of those jointly, or rely on heavily parametric approaches. Building on recent Dirichlet-Point processes literature, we introduce the Houston (Hidden Online User-Topic Network) model, that jointly considers all those features in a non-parametric unsupervised framework. It infers dynamic topic-dependent underlying diffusion networks in a continuous-time setting along with said topics. It is unsupervised; it considers an unlabeled stream of triplets shaped as (time of publication, information’s content, spreading entity) as input data. Online inference is conducted using a sequential Monte-Carlo algorithm that scales linearly with the size of the dataset. Our approach yields consequent improvements over existing baselines on both cluster recovery and subnetworks inference tasks.

A Bayesian Nonparametric Latent Space Approach to Modeling Evolving Communities in Dynamic Networks

Abstract: The evolution of communities in dynamic (time-varying) network data is a prominent topic of interest. A popular approach to understanding these dynamic networks is to embed the dyadic relations into a latent metric space. While methods for clustering with this approach exist for dynamic networks, they all assume a static community structure. This paper presents a Bayesian nonparametric model for dynamic networks that can model networks with evolving community structures. Our model extends existing latent space approaches by explicitly modeling the additions, deletions, splits, and mergers of groups with a hierarchical Dirichlet process hidden Markov model. Our proposed approach, the hierarchical Dirichlet process latent position cluster model (HDP-LPCM), incorporates transitivity, models both individual and group level aspects of the data, and avoids the computationally expensive selection of the number of groups required by most popular methods. We provide a Markov chain Monte Carlo estimation algorithm and demonstrate its ability to detect evolving community structure in a network of military alliances during the Cold War and a narrative network constructed from the Game of Thrones television series.

Dirichlet form approach to diffusions with discontinuous scale

Abstract: It is well known that a regular diffusion on an interval $I$ without killing inside is uniquely determined by a canonical scale function $s$ and a canonical speed measure $m$. Note that $s$ is a strictly increasing and continuous function and $m$ is a fully supported Radon measure on $I$. In this paper we will associate a general triple $(I,s,m)$, where $s$ is only assumed to be increasing and $m$ is not necessarily fully supported, to certain Markov processes by way of Dirichlet forms. A straightforward generalization of Dirichlet form associated to regular diffusion will be first put forward, and we will find out its corresponding continuous Markov process $\dot X$, for which the strong Markov property fails whenever $s$ is not continuous. Then by operating regular representations on Dirichlet form and Ray-Knight compactification on $\dot X$ respectively, the same unique desirable symmetric Hunt process associated to $(I,s,m)$ is eventually obtained. This Hunt process is homeomorphic to a quasidiffusion, which is known as a celebrated generalization of regular diffusion.

Bayesian estimation of clustered dependence structures in functional neuroconnectivity

Abstract: Motivated by the need to model joint dependence between regions of interest in functional neuroconnectivity for efficient inference, we propose a new sampling-based Bayesian clustering approach for covariance structures of high-dimensional Gaussian outcomes. The key technique is based on a Dirichlet process that clusters covariance sub-matrices into independent groups of outcomes, thereby naturally inducing sparsity in the whole brain connectivity matrix. A new split-merge algorithm is employed to improve the mixing of the Markov chain sampling that is shown empirically to recover both uniform and Dirichlet partitions with high accuracy. We investigate the empirical performance of the proposed method through extensive simulations. Finally, the proposed approach is used to group regions of interest into functionally independent groups in the Autism Brain Imaging Data Exchange participants with autism spectrum disorder and attention-deficit/hyperactivity disorder.

Properties of the Dirichlet type 3 distribution

Abstract: In this paper, we investigate the Dirichlet type 3 distribution. First, some main properties are elaborated and illustrated. Next, we set forward a representation which allows to compute many functionals in a closed form, making the Dirichlet type 3 distribution an exactly soluble model. Furthermore, we consider the Gibbs version of the Dirichlet type 3 distribution including selection. By using the representation mentioned above, we obtain the moment function of the geometrical average of the random variables according to the new distribution; special types of Bell polynomials are shown to be involved. Finally, we provide a concrete example to illustrate the performance of the Dirichlet type 3 distribution.

Truncated Poisson–Dirichlet approximation for Dirichlet process hierarchical models

Abstract: Abstract The Dirichlet process was introduced by Ferguson in 1973 to use with Bayesian nonparametric inference problems. A lot of work has been done based on the Dirichlet process, making it the most fundamental prior in Bayesian nonparametric statistics. Since the construction of Dirichlet process involves an infinite number of random variables, simulation-based methods are hard to implement, and various finite approximations for the Dirichlet process have been proposed to solve this problem. In this paper, we construct a new random probability measure called the truncated Poisson–Dirichlet process. It sorts the components of a Dirichlet process in descending order according to their random weights, then makes a truncation to obtain a finite approximation for the distribution of the Dirichlet process. Since the approximation is based on a decreasing sequence of random weights, it has a lower truncation error comparing to the existing methods using stick-breaking process. Then we develop a blocked Gibbs sampler based on Hamiltonian Monte Carlo method to explore the posterior of the truncated Poisson–Dirichlet process. This method is illustrated by the normal mean mixture model and Caron–Fox network model. Numerical implementations are provided to demonstrate the effectiveness and performance of our algorithm.

Graphical Dirichlet Process

Abstract: We consider the problem of clustering grouped data with possibly non-exchangeable groups whose dependencies can be characterized by a directed acyclic graph. To allow the sharing of clusters among the non-exchangeable groups, we propose a Bayesian nonparametric approach, termed graphical Dirichlet process, that jointly models the dependent group-specific random measures by assuming each random measure to be distributed as a Dirichlet process whose concentration parameter and based probability measure depend on those of its parent groups. The resulting joint stochastic process respects the Markov property of the directed acyclic graph that links the groups. We characterize the graphical Dirichlet process using a novel hypergraph representation as well as the stick-breaking representation, the restaurant-type representation, and the representation as a limit of a finite mixture model. We develop an efficient posterior inference algorithm and illustrate our model with simulations and a real grouped single-cell data.

Bayesian modeling via discrete nonparametric priors

Abstract: Abstract The availability of complex-structured data has sparked new research directions in statistics and machine learning. Bayesian nonparametrics is at the forefront of this trend thanks to two crucial features: its coherent probabilistic framework, which naturally leads to principled prediction and uncertainty quantification, and its infinite-dimensionality, which exempts from parametric restrictions and ensures full modeling flexibility. In this paper, we provide a concise overview of Bayesian nonparametrics starting from its foundations and the Dirichlet process, the most popular nonparametric prior. We describe the use of the Dirichlet process in species discovery, density estimation, and clustering problems. Among the many generalizations of the Dirichlet process proposed in the literature, we single out the Pitman–Yor process, and compare it to the Dirichlet process. Their different features are showcased with real-data illustrations. Finally, we consider more complex data structures, which require dependent versions of these models. One of the most effective strategies to achieve this goal is represented by hierarchical constructions. We highlight the role of the dependence structure in the borrowing of information and illustrate its effectiveness on unbalanced datasets.

Generalized Dirichlet Distribution Based on Confluent Hypergeometric Series

Abstract: Dirichlet distribution is a kind of high-dimensional continuous probability distribution, which has important applications in the fields of statistics, machine learning and bioinformatics. In this paper, based on gamma distribution we study two two-dimensional random variables. Then we derive the properties of these two two-dimensional random variables by using the properties of non-central gamma distribution and confluent hypergeometric series. From these properties, we find the two random variables follow generalized Dirichlet distributions. Applying hypergeometric series to Dirichlet distribution broadens the research of Dirichlet distribution.

Human Activity Recognition with an HMM-Based Generative Model

Abstract: Human activity recognition (HAR) has become an interesting topic in healthcare. This application is important in various domains, such as health monitoring, supporting elders, and disease diagnosis. Considering the increasing improvements in smart devices, large amounts of data are generated in our daily lives. In this work, we propose unsupervised, scaled, Dirichlet-based hidden Markov models to analyze human activities. Our motivation is that human activities have sequential patterns and hidden Markov models (HMMs) are some of the strongest statistical models used for modeling data with continuous flow. In this paper, we assume that emission probabilities in HMM follow a bounded–scaled Dirichlet distribution, which is a proper choice in modeling proportional data. To learn our model, we applied the variational inference approach. We used a publicly available dataset to evaluate the performance of our proposed model.

3D Multi-Views Object Classification Based on a Fully Generalized Dirichlet Allocation Model

Abstract: Comparing 3D objects based on their local features necessitates a substantial amount of computational resources. Recent studies have demonstrated that the combination of topic modeling and the Bag of Visual Words (BoVWs) approach can effectively capture useful and distinguishable object rep-resentations in a consistent way. One of the most common topic modeling approaches is LDA which is based on Dirichlet distribution priors, but it is limited by its incapacity in modeling topic correlations. Consequently, several extensions of LDA were proposed to solve this problem including GD-LDA, LGDA, and CVB-LGDA which have shown good results in discovering the semantic relationships between topics but are either suffering from incomplete generative processes assumptions that impact their inference efficiency or require high-computational power due to their complexity. In this paper, we introduce F-GDA, a fully Generalized Dirichlet Allocation model that is mainly derived from Generalized Dirichlet distributions for 3D objects recognition. Unlike GD-LDA which generalizes only the topics parameter, F-GDA generalizes all the model priors parameters, ensuring complete flexibility in the priors. Extensive experimental results have demonstrated the ability of our model to learn high-quality data representations of a real-world 3D Multi-views dataset ETH80 and also N15.