Showing papers by "Wentao Fan published in 2019"

Axially Symmetric Data Clustering Through Dirichlet Process Mixture Models of Watson Distributions

Abstract: This paper proposes a Bayesian nonparametric framework for clustering axially symmetric data. Our approach is based on a Dirichlet processes mixture model with Watson distributions, which can also be considered as the infinite Watson mixture model. In this paper, first, we extend the finite Watson mixture model into its infinite counterpart based on the framework of truncated Dirichlet process mixture model with a stick-breaking representation. Second, we propose a coordinate ascent mean-field variational inference algorithm that can effectively learn the parameters of our model with closed-form solutions; Third, to cope with a massive data set, we develop a stochastic variational inference algorithm to learn the proposed model through the method of stochastic gradient ascent; Finally, the proposed nonparametric Bayesian model is evaluated through simulated axially symmetric data sets and a real-world application, namely, gene expression data clustering.

Nonparametric Hierarchical Bayesian Models for Positive Data Clustering Based on Inverted Dirichlet-Based Distributions

Abstract: In this paper, we propose nonparametric hierarchical Bayesian models based on two inverted Dirichlet-based distributions and Pitman-Yor process for positive data features clustering. The choice of the inverted Dirichlet and the generalized inverted Dirichlet distributions is motivated by their flexibility and modeling capabilities when dealing with this kind of data, while deploying the Pitman-Yor process prior is justified by its power-law behavior, which makes it a natural choice in real-life application compared with Dirichlet processes for instance. The inference for the resulting models takes into account the challenging problem of feature weighting/selection and is conducted under a Bayesian setting by means of the recently proposed stochastic variational Bayes technique. The efficacy and merits of the proposed approaches are examined using the synthetic data and a challenging real-life application that concerns video background subtraction.

Semi-Supervised Semantic-Preserving Hashing for Efficient Cross-Modal Retrieval

Abstract: Cross-modal hashing has recently gained significant popularity to facilitate retrieval across different modalities. With limited label available, this paper presents a novel Semi-Supervised Semantic-Preserving Hashing (S3PH) for flexible cross-modal retrieval. In contrast to most semi-supervised cross-modal hashing works that need to predict the label of unlabeled data, our proposed approach groups the labeled and unlabeled data together, and integrates the relaxed latent subspace learning and semantic-preserving regularization across different modalities. Accordingly, an efficient relaxed objective function is proposed to learn the latent subspaces for both labeled and unlabeled data. Further, an orthogonal rotation matrix is efficiently learned to transform the latent subspace to hash space by minimizing the quantization error. Without sacrificing the retrieval performance, the proposed S3PH method can benefit various kinds of retrieval tasks, i.e., unsupervised, semi-supervised and supervised. Experimental results compared with several competitive algorithms show the effectiveness of the proposed method and its superiority over state-of-the-arts.

Localization-Aware Meta Tracker Guided With Adversarial Features

Abstract: Deep learning has recently shown great potentials in learning powerful features for visual tracking. However, most deep learning-based trackers neglect localization accuracy in the evaluation process of candidates. What’s more, they usually over-rely on the discriminative features in a single frame in the training process. Consequently, they may fail when the discriminative features are occluded or changed in the tracking phase. In this paper, we propose a novel localization-aware meta tracker (LMT) guided with adversarial features to address the above issues. First of all, we design a novel intersection over union guided method to effectively balance the problem of classification and localization accuracy. To further improve the robustness of our classifier, we creatively use adversarial features during offline training phase. Those adversarial features can effectively guide the classifier in learning how to better deal with the situation where the discriminative features are occluded or changed. Finally, benefiting from meta learning, our algorithm only needs to perform one iterative update on the first frame and it can perform well on the tracking sceneries. The extensive experiments demonstrate the outstanding performance of our LMT compared with the state-of-the-art trackers on three benchmarks: OTB-2015, VOT-2016, and VOT-2018.

A nonparametric Bayesian learning model using accelerated variational inference and feature selection

Abstract: Developing effective machine learning methods for multimedia data modeling continues to challenge computer vision scientists. The capability of providing effective learning models can have significant impact on various applications. In this work, we propose a nonparametric Bayesian approach to address simultaneously two fundamental problems, namely clustering and feature selection. The approach is based on infinite generalized Dirichlet (GD) mixture models constructed through the framework of Dirichlet process and learned using an accelerated variational algorithm that we have developed. Furthermore, we extend the proposed approach using another nonparametric Bayesian prior, namely Pitman---Yor process, to construct the infinite generalized Dirichlet mixture model. Our experiments, which were conducted through synthetic data sets, the clustering analysis of real-world data sets and a challenging application, namely automatic human action recognition, indicate that the proposed framework provides good modeling and generalization capabilities.

Learning of Multivariate Beta Mixture Models via Entropy-based component splitting

Abstract: Finite mixture models are progressively employed in various fields of science due to their high potential as inference engines to model multimodal and complex data. To develop them, we face some crucial issues such as choosing proper distributions with enough flexibility to well-fit the data. To learn our model, two other significant challenges, namely, parameter estimation and defining model complexity have to be addressed. Some methods such as maximum likelihood and Bayesian inference have been widely considered to tackle the first problem and both have some drawbacks such as local maxima or high computational complexity. Simultaneously, the proper number of components was determined with some approaches such as minimum message length. In this work, multivariate Beta mixture models have been deployed thanks to their flexibility and we propose a novel variational inference via an entropy-based splitting method. The performance of this approach is evaluated on real-world applications, namely, breast tissue texture classification, cytological breast data analysis, cell image categorization and age estimation.

Simultaneous clustering and feature selection via nonparametric Pitman–Yor process mixture models

Abstract: Mixture models constitute one of the most important machine learning approaches. Indeed, they can be considered as the workhorse of generative machine learning. The majority of existing works consider mixtures of Gaussians. Unlike these works, this paper concentrates on nonparametric Bayesian models with Dirichlet-based mixtures. In particular, we consider the case when a Pitman–Yor process prior is adopted. Two central problems when considering such mixtures can be regarded as selecting ‘meaningful’ (or relevant) features and estimating the model’s parameters. We develop an efficient algorithm for model inference, based on the collapsed variational Bayes framework with 0th-order Taylor approximation. The merits and efficacy of the proposed nonparametric Bayesian model are demonstrated via challenging applications that concern real-world data clustering and 3D objects recognition.

Positive Sequential Data Modeling Using Continuous Hidden Markov Models Based on Inverted Dirichlet Mixtures

Abstract: The hidden Markov model (HMM) has long been one of the most commonly used probability graph models for modeling sequential or time series data. It has been widely used in many fields ranging from speech recognition, face recognition, anomaly detection, to gene function prediction. In this paper, we theoretically propose a variant of the continuous HMM for modeling positive sequential data which are naturally generated in many real-life applications. In contrast with conventional HMMs which often use Gaussian distributions or Gaussian mixture models as the emission probability density, we adopt the inverted Dirichlet mixture model as the emission density to build the HMM. The consideration of inverted Dirichlet mixture model in our case is motivated by its superior modeling capability over Gaussian mixture models for modeling positive data according to several recent studies. In addition, we develop a convergence-guaranteed approach to learning the proposed inverted Dirichlet-based HMM through variational Bayes inference. The effectiveness of the proposed HMM is validated through both synthetic data sets and a real-world application regarding anomaly network intrusion detection. Based on the experimental results, the proposed inverted Dirichlet-based HMM is able to achieve the detection accuracy rates that are about 4%~9% higher than those ones obtained by the compared approaches.

Variational Learning for Finite Generalized Inverted Dirichlet Mixture Models with a Component Splitting Approach

Abstract: Mixture models play a crucial role in pattern recognition methods based on clustering. In this paper, we propose a finite generalized inverted Dirichlet mixture model with a variational learning method for parameter estimation. The highlight of our model is the component splitting approach which handles the problem of model selection in an incremental fashion within the variational framework. Efficiency of proposed model is tested for image categorization tasks.

Unsupervised Image Categorization Based on Variational Autoencoder and Student’s-T Mixture Model

Abstract: In this work, a novel generative robust image catego-rization approach is developed based on variational autoencoder (VAE) and Student’s-T Mixture Model (STMM). The network structure composed of VAE, STMM and Convolutional Neural Network (CNN) generates data. More specifically, first, a cluster is chosen using the STMM. Then, a latent representation is extracted from the selected cluster through a CNN encoder. After that, an observation is generated based on another CNN through a decoding process. The proposed model is learned through variational inference where the Evidence Lower Bound is optimized according to Stochastic Gradient Descent(SGD) and the reparameterization trick. Based on our experimental results, the proposed generative clustering approach is able to outperform classical clustering approaches (e.g. K-means, Gaussian Mixture Models) and other related generative clustering approaches. Furthermore, we show that our generative model is able to generate highly realistic samples without using any supervised information during training.

Unsupervised Variational Learning of Finite Generalized Inverted Dirichlet Mixture Models with Feature Selection and Component Splitting

Abstract: Variational learning of mixture models has proved to be effective in recent research. In this paper, we propose a generalized inverted Dirichlet based mixture model with an incremental variational algorithm. We incorporate feature selection and a component splitting approach for model selection within the variational framework. This helps us estimate the complexity of the data efficiently concomitantly eliminating the irrelevant features. We validate our model with two challenging applications; image categorization and dynamic texture categorization.