Statistical node clustering in discrete time dynamic networks is an emerging field that raises many challenges. Here, we explore statistical properties and frequentist inference in a model that combines a stochastic block model (SBM) for its static part with independent Markov chains for the evolution of the nodes groups through time. We model binary data as well as weighted dynamic random graphs (with discrete or continuous edges values). Our approach, motivated by the importance of controlling for label switching issues across the different time steps, focuses on detecting groups characterized by a stable within group connectivity behavior. We study identifiability of the model parameters, propose an inference procedure based on a variational expectation maximization algorithm as well as a model selection criterion to select for the number of groups. We carefully discuss our initialization strategy which plays an important role in the method and compare our procedure with existing ones on synthetic datasets. We also illustrate our approach on dynamic contact networks, one of encounters among high school students and two others on animal interactions. An implementation of the method is available as a R package called dynsbm.

Statistical clustering of temporal networks through a dynamic stochastic block model

Significant progress has been made recently on theoretical analysis of estimators for the stochastic block model (SBM). In this paper, we consider the multi-graph SBM, which serves as a foundation for many application settings including dynamic and multi-layer networks. We explore the asymptotic properties of two estimators for the multi-graph SBM, namely spectral clustering and the maximum-likelihood estimate (MLE), as the number of layers of the multigraph increases. We derive sufficient conditions for consistency of both estimators and propose a variational approximation to the MLE that is computationally feasible for large networks. We verify the sufficient conditions via simulation and demonstrate that they are practical. In addition, we apply the model to two real data sets: a dynamic social network and a multi-layer social network with several types of relations.

/pdf/consistent-estimation-of-dynamic-and-multi-layer-block-3nnjcd2aek.pdf

Consistent estimation of dynamic and multi-layer block models

Visual text analytics has recently emerged as one of the most prominent topics in both academic research and the commercial world. To provide an overview of the relevant techniques and analysis tasks, as well as the relationships between them, we comprehensively analyzed 263 visualization papers and 4,346 mining papers published between 1992-2017 in two fields: visualization and text mining. From the analysis, we derived around 300 concepts (visualization techniques, mining techniques, and analysis tasks) and built a taxonomy for each type of concept. The co-occurrence relationships between the concepts were also extracted. Our research can be used as a stepping-stone for other researchers to 1) understand a common set of concepts used in this research topic; 2) facilitate the exploration of the relationships between visualization techniques, mining techniques, and analysis tasks; 3) understand the current practice in developing visual text analytics tools; 4) seek potential research opportunities by narrowing the gulf between visualization and mining techniques based on the analysis tasks; and 5) analyze other interdisciplinary research areas in a similar way. We have also contributed a web-based visualization tool for analyzing and understanding research trends and opportunities in visual text analytics.

/pdf/bridging-text-visualization-and-mining-a-task-driven-survey-44katda0fb.pdf

Bridging Text Visualization and Mining: A Task-Driven Survey

The focus of this work is on developing probabilistic models for temporal activity of users in social networks (e.g., posting and tweeting) by incorporating the social network influence as perceived by the user. Although prior work in this area has developed sophisticated models for user activity, these models either ignore social network influence completely or incorporate it in an implicit manner. We overcome the nontransparency of the network in the model at the individual scale by proposing a coupled hidden Markov model (HMM), where each user’s activity evolves according to a Markov chain with a hidden state that is influenced by the collective activity of the friends of the user. We develop generalized Baum–Welch and Viterbi algorithms for parameter learning and state estimation for the proposed framework. We then validate the proposed model using a significant corpus of user activity on Twitter. Our numerical studies show that with sufficient observations to ensure accurate model learning, the proposed framework explains the observed data better than either a renewal process-based model or a conventional (uncoupled) HMM. We also demonstrate the utility of the proposed approach in predicting the time to the next tweet. Finally, clustering in the model parameter space is shown to result in distinct natural clusters of users characterized by the interaction dynamic between a user and his network.

Modeling Temporal Activity Patterns in Dynamic Social Networks

There has been great interest in recent years on statistical models for dynamic networks. In this paper, I propose a stochastic block transition model (SBTM) for dynamic networks that is inspired by the well-known stochastic block model (SBM) for static networks and previous dynamic extensions of the SBM. Unlike most existing dynamic network models, it does not make a hidden Markov assumption on the edge-level dynamics, allowing the presence or absence of edges to directly influence future edge probabilities while retaining the interpretability of the SBM. I derive an approximate inference procedure for the SBTM and demonstrate that it is significantly better at reproducing durations of edges in real social network data.

Stochastic Block Transition Models for Dynamic Networks

Current Bayesian models for dynamic social network data have focused on modelling the influence of evolving unobserved structure on observed social interactions. However, an understanding of how observed social relationships from the past affect future unobserved structure in the network has been neglected. In this paper, we introduce a new probabilistic model for capturing this phenomenon, which we call latent feature propagation, in social networks. We demonstrate our model's capability for inferring such latent structure in varying types of social network datasets, and experimental studies show this structure achieves higher predictive performance on link prediction and forecasting tasks.

/pdf/dynamic-probabilistic-models-for-latent-feature-propagation-rm39gkt6am.pdf

Dynamic Probabilistic Models for Latent Feature Propagation in Social Networks

We characterize the combinatorial structure of conditionally-i.i.d. sequences of negative binomial processes with a common beta process base measure. In Bayesian nonparametric applications, such processes have served as models for latent multisets of features underlying data. Analogously, random subsets arise from conditionally-i.i.d. sequences of Bernoulli processes with a common beta process base measure, in which case the combinatorial structure is described by the Indian buffet process. Our results give a count analogue of the Indian buffet process, which we call a negative binomial Indian buffet process. As an intermediate step toward this goal, we provide a construction for the beta negative binomial process that avoids a representation of the underlying beta process base measure. We describe the key Markov kernels needed to use a NB-IBP representation in a Markov Chain Monte Carlo algorithm targeting a posterior distribution.

/pdf/the-combinatorial-structure-of-beta-negative-binomial-ek0dkhhqr4.pdf

The combinatorial structure of beta negative binomial processes

The collection of data from a personal digital device to characterize current health conditions and behaviors that determine how an individual's health will evolve has been called digital phenotyping. In this paper, we describe the development of and early experiences with a comprehensive digital phenotyping platform: Health Outcomes through Positive Engagement and Self-Empowerment (HOPES). HOPES is based on the open-source Beiwe platform but adds a wider range of data collection, including the integration of wearable devices and further sensor collection from smartphones. Requirements were partly derived from a concurrent clinical trial for schizophrenia that required the development of significant capabilities in HOPES for security, privacy, ease of use, and scalability, based on a careful combination of public cloud and on-premises operation. We describe new data pipelines to clean, process, present, and analyze data. This includes a set of dashboards customized to the needs of research study operations and clinical care. A test use case for HOPES was described by analyzing the digital behavior of 22 participants during the SARS-CoV-2 pandemic.

/pdf/hopes-an-integrative-digital-phenotyping-platform-for-data-2zh8re9y6i.pdf

HOPES: An Integrative Digital Phenotyping Platform for Data Collection, Monitoring, and Machine Learning.

We investigate a class of feature allocation models that generalize the Indian buffet process and are parameterized by Gibbs-type random measures. Two existing classes are contained as special cases: the original two-parameter Indian buffet process, corresponding to the Dirichlet process, and the stable (or three-parameter) Indian buffet process, corresponding to the Pitman--Yor process. Asymptotic behavior of the Gibbs-type partitions, such as power laws holding for the number of latent clusters, translates into analogous characteristics for this class of Gibbs-type feature allocation models. Despite containing several different distinct subclasses, the properties of Gibbs-type partitions allow us to develop a black-box procedure for posterior inference within any subclass of models. Through numerical experiments, we compare and contrast a few of these subclasses and highlight the utility of varying power-law behaviors in the latent features.

/pdf/gibbs-type-indian-buffet-processes-43jpns9wk6.pdf

Gibbs-type Indian buffet processes

We implement gradient-based variational inference routines for Wishart and inverse Wishart processes, which we apply as Bayesian models for the dynamic, heteroskedastic covariance matrix of a multivariate time series. The Wishart and inverse Wishart processes are constructed from i.i.d. Gaussian processes, existing variational inference algorithms for which form the basis of our approach. These methods are easy to implement as a black-box and scale favorably with the length of the time series, however, they fail in the case of the Wishart process, an issue we resolve with a simple modification into an additive white noise parameterization of the model. This modification is also key to implementing a factored variant of the construction, allowing inference to additionally scale to high-dimensional covariance matrices. Through experimentation, we demonstrate that some (but not all) model variants outperform multivariate GARCH when forecasting the covariances of returns on financial instruments.

/pdf/scalable-bayesian-dynamic-covariance-modeling-with-20yvcdxffc.pdf

Creighton Heaukulani

Papers

Dynamic Probabilistic Models for Latent Feature Propagation in Social Networks

The combinatorial structure of beta negative binomial processes

HOPES: An Integrative Digital Phenotyping Platform for Data Collection, Monitoring, and Machine Learning.

Gibbs-type Indian buffet processes

Scalable Bayesian dynamic covariance modeling with variational Wishart and inverse Wishart processes