Showing papers on "Mixture model published in 2021"

Robot Programming by Demonstration: A Probabilistic Approach

Abstract: ACKNOWLEDGMENT INTRODUCTION Contributions Organization of the book Review of Robot Programming by Demonstration (PBD) Current state of the art in PbD SYSTEM ARCHITECTURE Illustration of the proposed probabilistic approach Encoding of motion in a Gaussian Mixture Model (GMM) Encoding of motion in Hidden Markov Model (HMM) Reproduction through Gaussian Mixture Regression (GMR) Reproduction by considering multiple constraints Learning of model parameters Reduction of dimensionality and latent space projection Model selection and initialization Regularization of GMM parameters Use of prior information to speed up the learning process Extension to mixture models of varying density distributions Summary of the chapter COMPARISON AND OPTIMIZATION OF THE PARAMETERS Optimal reproduction of trajectories through HMM and GMM/GMR Optimal latent space of motion Optimal selection of the number of Gaussians Robustness evaluation of the incremental learning process HANDLING OF CONSTRAINTS IN JOINT SPACE AND TASK SPACE Inverse kinematics Handling of task constraints in joint spaceexperiment with industrial robot Handling of task constraints in latent spaceexperiment with humanoid robot EXTENSION TO DYNAMICAL SYSTEM AND HANDLING OF PERTURBATIONS Proposed dynamical system Influence of the dynamical system parameters Experimental setup Experimental results TRANSFERRING SKILLS THROUGH ACTIVE TEACHING METHODS Experimental setup Experimental results Roles of an active teaching scenario USING SOCIAL CUES TO SPEED UP THE LEARNING PROCESS Experimental setup Experimental results DISCUSSION, FUTURE WORK AND CONCLUSIONS Advantages of the proposed approach Failures and limitations of the proposed approach Further issues Final words REFERENCES INDEX

Fuzzy C-Means clustering algorithm for data with unequal cluster sizes and contaminated with noise and outliers: Review and development

Abstract: Clustering algorithms aim at finding dense regions of data based on similarities and dissimilarities of data points. Noise and outliers contribute to the computational procedure of the algorithms as well as the actual data points that leads to inaccurate and misplaced cluster centers. This problem also arises when sizes of the clusters are different that moves centers of small clusters towards large clusters. Mass of the data points is important as well as their location in engineering and physics where non-uniform mass distribution results displacement of the cluster centers towards heavier clusters even if sizes of the clusters are identical and the data are noise-free. Fuzzy C-Means (FCM) algorithm that suffers from these problems is the most popular fuzzy clustering algorithm and has been subject of numerous researches and developments though improvements are still marginal. This work revises the FCM algorithm to make it applicable to data with unequal cluster sizes, noise and outliers, and non-uniform mass distribution. Revised FCM (RFCM) algorithm employs adaptive exponential functions to eliminate impacts of noise and outliers on the cluster centers and modifies constraint of the FCM algorithm to prevent large or heavier clusters from attracting centers of small clusters. Several algorithms are reviewed and their mathematical structures are discussed in the paper including Possibilistic Fuzzy C-Means (PFCM), Possibilistic C-Means (PCM), Robust Fuzzy C-Means (FCM-σ), Noise Clustering (NC), Kernel Fuzzy C-Means (KFCM), Intuitionistic Fuzzy C-Means (IFCM), Robust Kernel Fuzzy C-Mean (KFCM-σ), Robust Intuitionistic Fuzzy C-Means (IFCM-σ), Kernel Intuitionistic Fuzzy C-Means (KIFCM), Robust Kernel Intuitionistic Fuzzy C-Means (KIFCM-σ), Credibilistic Fuzzy C-Means (CFCM), Size-insensitive integrity-based Fuzzy C-Means (siibFCM), Size-insensitive Fuzzy C-Means (csiFCM), Subtractive Clustering (SC), Density Based Spatial Clustering of Applications with Noise (DBSCAN), Gaussian Mixture Models (GMM), Spectral clustering, and Outlier Removal Clustering (ORC). Some of these algorithms are suitable for noisy data and some others are designed for data with unequal clusters. The study shows that the RFCM algorithm works for both cases and outperforms the both categories of the algorithms.

Learning Accurate Dense Correspondences and When to Trust Them

Abstract: Establishing dense correspondences between a pair of images is an important and general problem. However, dense flow estimation is often inaccurate in the case of large displacements or homogeneous regions. For most applications and down-stream tasks, such as pose estimation, image manipulation, or 3D reconstruction, it is crucial to know when and where to trust the estimated matches.In this work, we aim to estimate a dense flow field relating two images, coupled with a robust pixel-wise confidence map indicating the reliability and accuracy of the prediction. We develop a flexible probabilistic approach that jointly learns the flow prediction and its uncertainty. In particular, we parametrize the predictive distribution as a constrained mixture model, ensuring better modelling of both accurate flow predictions and outliers. Moreover, we develop an architecture and training strategy tailored for robust and generalizable uncertainty prediction in the context of self-supervised training. Our approach obtains state- of-the-art results on multiple challenging geometric matching and optical flow datasets. We further validate the usefulness of our probabilistic confidence estimation for the task of pose estimation. Code and models are available at https://github.com/PruneTruong/PDCNet.

Deep learning-based mixed-dimensional Gaussian mixture model for characterizing variability in cryo-EM.

Abstract: Structural flexibility and/or dynamic interactions with other molecules is a critical aspect of protein function. Cryogenic electron microscopy (cryo-EM) provides direct visualization of individual macromolecules sampling different conformational and compositional states. While numerous methods are available for computational classification of discrete states, characterization of continuous conformational changes or large numbers of discrete state without human supervision remains challenging. Here we present e2gmm, a machine learning algorithm to determine a conformational landscape for proteins or complexes using a three-dimensional Gaussian mixture model mapped onto two-dimensional particle images in known orientations. Using a deep neural network architecture, e2gmm can automatically resolve the structural heterogeneity within the protein complex and map particles onto a small latent space describing conformational and compositional changes. This system presents a more intuitive and flexible representation than other manifold methods currently in use. We demonstrate this method on both simulated data and three biological systems to explore compositional and conformational changes at a range of scales. The software is distributed as part of EMAN2.

Moving Vehicle Detection and Classification Using Gaussian Mixture Model and Ensemble Deep Learning Technique

Abstract: In recent decades, automatic vehicle classification plays a vital role in intelligent transportation systems and visual traffic surveillance systems. Especially in countries that imposed a lockdown (mobility restrictions help reduce the spread of COVID-19), it becomes important to curtail the movement of vehicles as much as possible. For an effective visual traffic surveillance system, it is essential to detect vehicles from the images and classify the vehicles into different types (e.g., bus, car, and pickup truck). Most of the existing research studies focused only on maximizing the percentage of predictions, which have poor real-time performance and consume more computing resources. To highlight the problems of classifying imbalanced data, a new technique is proposed in this research article for vehicle type classification. Initially, the data are collected from the Beijing Institute of Technology Vehicle Dataset and the MIOvision Traffic Camera Dataset. In addition, adaptive histogram equalization and the Gaussian mixture model are implemented for enhancing the quality of collected vehicle images and to detect vehicles from the denoised images. Then, the Steerable Pyramid Transform and the Weber Local Descriptor are employed to extract the feature vectors from the detected vehicles. Finally, the extracted features are given as the input to an ensemble deep learning technique for vehicle classification. In the simulation phase, the proposed ensemble deep learning technique obtained 99.13% and 99.28% of classification accuracy on the MIOvision Traffic Camera Dataset and the Beijing Institute of Technology Vehicle Dataset. The obtained results are effective compared to the standard existing benchmark techniques on both datasets.

Self-Paced Dynamic Infinite Mixture Model for Fatigue Evaluation of Pilots' Brains.

Abstract: Current brain cognitive models are insufficient in handling outliers and dynamics of electroencephalogram (EEG) signals. This article presents a novel self-paced dynamic infinite mixture model to infer the dynamics of EEG fatigue signals. The instantaneous spectrum features provided by ensemble wavelet transform and Hilbert transform are extracted to form four fatigue indicators. The covariance of log likelihood of the complete data is proposed to accurately identify similar components and dynamics of the developed mixture model. Compared with its seven peers, the proposed model shows better performance in automatically identifying a pilot's brain workload.

Self-Supervised Learning on 3D Point Clouds by Learning Discrete Generative Models

Abstract: While recent pre-training tasks on 2D images have proven very successful for transfer learning, pre-training for 3D data remains challenging. In this work, we introduce a general method for 3D self-supervised representation learning that 1) remains agnostic to the underlying neural network architecture, and 2) specifically leverages the geometric nature of 3D point cloud data. The proposed task softly segments 3D points into a discrete number of geometric partitions. A self-supervised loss is formed under the interpretation that these soft partitions implicitly parameterize a latent Gaussian Mixture Model (GMM), and that this generative model establishes a data likelihood function. Our pretext task can therefore be viewed in terms of an encoder-decoder paradigm that squeezes learned representations through an implicitly defined parametric discrete generative model bottleneck. We show that any existing neural network architecture designed for supervised point cloud segmentation can be repurposed for the proposed unsupervised pretext task. By maximizing data likelihood with respect to the soft partitions formed by the unsupervised point-wise segmentation network, learned representations are encouraged to contain compositionally rich geometric information. In tests, we show that our method naturally induces semantic separation in feature space, resulting in state-of-the-art performance on downstream applications like model classification and semantic segmentation.

SNMF-Net: Learning a Deep Alternating Neural Network for Hyperspectral Unmixing

Abstract: Hyperspectral unmixing is recognized as an important tool to learn the constituent materials and corresponding distribution in a scene. The physical spectral mixture model is always important to tackle this problem because of its highly ill-posed nature. In this article, we introduce a linear spectral mixture model (LMM)-based end-to-end deep neural network named SNMF-Net for hyperspectral unmixing. SNMF-Net shares an alternating architecture and benefits from both model-based methods and learning-based methods. On the one hand, SNMF-Net is of high physical interpretability as it is built by unrolling Lp sparsity constrained nonnegative matrix factorization (Lp-NMF) model belonging to LMM families. On the other hand, all the parameters and submodules of SNMF-Net can be seamlessly linked with the alternating optimization algorithm of Lp-NMF and unmixing problem. This enables us to reasonably integrate the prior knowledge on unmixing, the optimization algorithm, and the sparse representation theory into the network for robust learning, so as to improve unmixing. Experimental results on the synthetic and real-world data show the advantages of the proposed SNMF-Net over many state-of-the-art methods.

IQDet: Instance-wise Quality Distribution Sampling for Object Detection

Abstract: We propose a dense object detector with an instance-wise sampling strategy, named IQDet. Instead of using human prior sampling strategies, we first extract the regional feature of each ground-truth to estimate the instance-wise quality distribution. According to a mixture model in spatial dimensions, the distribution is more noise-robust and adapted to the semantic pattern of each instance. Based on the distribution, we propose a quality sampling strategy, which automatically selects training samples in a probabilistic manner and trains with more high-quality samples. Extensive experiments on MS COCO show that our method steadily improves baseline by nearly 2.4 AP without bells and whistles. Moreover, our best model achieves 51.6 AP, outperforming all existing state-of-the-art one-stage detectors and it is completely cost-free in inference time.

On posterior contraction of parameters and interpretability in Bayesian mixture modeling

Abstract: We study posterior contraction behaviors for parameters of interest in the context of Bayesian mixture modeling, where the number of mixing components is unknown while the model itself may or may not be correctly specified. Two representative types of prior specification will be considered: one requires explicitly a prior distribution on the number of mixture components, while the other places a nonparametric prior on the space of mixing distributions. The former is shown to yield an optimal rate of posterior contraction on the model parameters under minimal conditions, while the latter can be utilized to consistently recover the unknown number of mixture components, with the help of a fast probabilistic post-processing procedure. We then turn the study of these Bayesian procedures to the realistic settings of model misspecification. It will be shown that the modeling choice of kernel density functions plays perhaps the most impactful roles in determining the posterior contraction rates in the misspecified situations. Drawing on concrete posterior contraction rates established in this paper we wish to highlight some aspects about the interesting tradeoffs between model expressiveness and interpretability that a statistical modeler must negotiate in the rich world of mixture modeling.

Useful life prediction based on wavelet packet decomposition and two-dimensional convolutional neural network for lithium-ion batteries

Abstract: It is important to know the replace time for reducing the lithium-ion battery risk and assessing its reliability. For this purpose, the remaining useful life (RUL) can play an important role in the prognostics and health management of battery to solve the inaccurate prediction issue. The existing RUL prediction techniques for lithium-ion batteries are inefficient for learning long-term dependencies among capacity degradations. In this work, a new forecasting approach is proposed based on wavelet packet decomposition, two-dimensional convolutional neural network, and adaptive multiple error corrections. In this model, the bivariate Dirichlet mixture model is considered to make the heteroscedasticity of the unpredictable residuals signal based non-parametric distribution. To show the validity of the proposed model, the experimental data are considered based on Continental Europe and NASA Ames Prognostics Center of Excellence battery datasets. The obtained numerical analysis presents an accurate forecasting model. Different comparisons with the well-known models are made to show the validity of the suggested approach, which proves the superiority and forecasting stability of the proposed model.

Semantic Recognition of Human-Object Interactions via Gaussian-Based Elliptical Modeling and Pixel-Level Labeling

Abstract: Human-Object Interaction (HOI) recognition, due to its significance in many computer vision-based applications, requires in-depth and meaningful details from image sequences. Incorporating semantics in scene understanding has led to a deep understanding of human-centric actions. Therefore, in this research work, we propose a semantic HOI recognition system based on multi-vision sensors. In the proposed system, the de-noised RGB and depth images, via Bilateral Filtering (BLF), are segmented into multiple clusters using a Simple Linear Iterative Clustering (SLIC) algorithm. The skeleton is then extracted from segmented RGB and depth images via Euclidean Distance Transform (EDT). Human joints, extracted from the skeleton, provide the annotations for accurate pixel-level labeling. An elliptical human model is then generated via a Gaussian Mixture Model (GMM). A Conditional Random Field (CRF) model is trained to allocate a specific label to each pixel of different human body parts and an interaction object. Two semantic feature types that are extracted from each labeled body part of the human and labelled objects are: Fiducial points and 3D point cloud. Features descriptors are quantized using Fisher’s Linear Discriminant Analysis (FLDA) and classified using K-ary Tree Hashing (KATH). In experimentation phase the recognition accuracy achieved with the Sports dataset is 92.88%, with the Sun Yat-Sen University (SYSU) 3D HOI dataset is 93.5% and with the Nanyang Technological University (NTU) RGB+D dataset it is 94.16%. The proposed system is validated via extensive experimentation and should be applicable to many computer-vision based applications such as healthcare monitoring, security systems and assisted living etc.

Spatial clustering and modelling for landslide susceptibility mapping in the north of the Kathmandu Valley, Nepal

Abstract: In this article, we propose and test alternative sampling strategies based on clustering distribution concepts to increase the efficiency of the landslide susceptibility model outcomes, instead of common random selection method for training and testing samples. To that end, we prepared a comprehensive landslide inventory and used six unsupervised clustering algorithms (K-means, K-medoids, hierarchical cluster (HC) analysis, expectation–maximization using Gaussian mixture models (EM/GMM), affinity propagation, and mini batch K-means) to generate six different training datasets. After getting the cluster pattern in each technique, we classified it into 70% and 30% for training and testing samples, respectively. We generated an additional training dataset using random selection procedure to test the hypothesis. The EM/GMM model exhibited the highest accuracy than the other methods. The findings confirm the hypothesis and recommend investing in natural distribution of landslides incident, as training concepts, instead of random sampling.

Exploring the forecasting approach for road accidents: Analytical measures with hybrid machine learning

Abstract: Urban traffic forecasting models generally follow either a Gaussian Mixture Model (GMM) or a Support Vector Classifier (SVC) to estimate the features of potential road accidents. Although SVC can provide good performances with less data than GMM, it incurs a higher computational cost. This paper proposes a novel framework that combines the descriptive strength of the Gaussian Mixture Model with the high-performance classification capabilities of the Support Vector Classifier. A new approach is presented that uses the mean vectors obtained from the GMM model as input to the SVC. Experimental results show that the approach compares very favorably with baseline statistical methods.

Stabilizing Invertible Neural Networks Using Mixture Models

Abstract: In this paper, we analyze the properties of invertible neural networks, which provide a way of solving inverse problems. Our main focus lies on investigating and controlling the Lipschitz constants of the corresponding inverse networks. Without such an control, numerical simulations are prone to errors and not much is gained against traditional approaches. Fortunately, our analysis indicates that changing the latent distribution from a standard normal one to a Gaussian mixture model resolves the issue of exploding Lipschitz constants. Indeed, numerical simulations confirm that this modification leads to significantly improved sampling quality in multimodal applications.

A Comparison of Statistical Tools for Identifying Modality in Body Mass Distributions

Abstract: The assessment of modality or \bumps" in distributions is of in- terest to scientists in many areas. We compare the performance of four statistical methods to test for departures from unimodality in simulations, and further illustrate the four methods using well-known ecological datasets on body mass published by Holling in 1992 to illustrate their advantages and disadvantages. Silverman's kernel density method was found to be very conservative. The excess mass test and a Bayesian mixture model approach showed agreement among the data sets, whereas Hall and York's test pro- vided strong evidence for the existence of two or more modes in all data sets. The Bayesian mixture model also provided a way to quantify the un- certainty associated with the number of modes. This work demonstrates the inherent richness of animal body mass distributions but also the diculties for characterizing it, and ultimately understanding the processes underlying them.

Mixture Density Networks-based Knock Simulator

Abstract: The engine knock simulator is useful for the evaluation of the feedback knock controllers and also the calibration of the feedforward control input without experiments in spark-ignition engines. This paper proposes a Mixture Density Network(MDN)-based statistical simulator of the engine knock for spark-ignition engines. The simulator can output the simulated knock intensity by the operating condition, which has a consistent probability distribution with the real engine. The statistical analysis is conducted based on the experimental data. According to the analysis results, several important properties about the knock intensity have been revealed. The logarithm of knock intensity is independent and identically distributed under an identical operating condition, whose probability distribution can be approximated by Gaussian Mixture Model(GMM). The parameter vector of the GMM is a function of the engine's operation condition. Based on these statistical properties of engine knock, we formulate the problem of establishing a statistical simulator, which includes two sub-problems. The first one is how to approximate the function from the operating condition to the parameters of the GMM. The second one is how to output the simulated random data of logarithm of knock intensity that obeys a given distribution. The MDN and the accept-reject algorithm are applied to solve the two sub-problem, respectively. Finally, we conducted experimental data-based validations to verify the proposed method.

Machine Learning for Automatic Processing of Modal Analysis in Damage Detection of Bridges

Abstract: Autonomous structural health monitoring (SHM) of a large number of bridges became a topic of paramount importance for maintenance purposes and safety reasons. This article proposes a set of machine learning (ML) tools to perform automatic detection of anomalies in a bridge structure from vibrational data. As a case study, we considered the Z-24 bridge for which an extensive database of accelerometric data is available. The proposed framework starts from the stabilization diagram obtained through operational modal analysis (OMA) to perform the clustering of modal frequencies and their tracking by density-based time-domain filtering. The features extracted are then fed to a one-class classification (OCC) algorithm to perform anomaly detection. In particular, we propose two new anomaly detectors, namely, one-class classifier neural network (OCCNN) and OCCNN2, that find the normal class (the boundary of the features space in normal operating conditions) through a two-step approach: coarse and fine boundary estimate. The detection algorithms are then compared with known methods based on the principal component analysis (PCA), the kernel PCA (KPCA), the Gaussian mixture model (GMM), and the autoassociative neural network (ANN). The proposed OCCNN solution presents increased accuracy and F1 score over conventional algorithms, without the need to set critical parameters, while OCCNN2 provides the best performance in terms of F1 score, accuracy, and responsiveness.

Tensor RNN With Bayesian Nonparametric Mixture for Radar HRRP Modeling and Target Recognition

Abstract: To deal with the temporal dependence between range cells in high resolution range profile (HRRP), dynamic methods, especially recurrent neural network (RNN), have been employed to extract features for target recognition. However, RNN has difficulty in complex and diverse sequence modeling problems as it ignores non-stationary sequential relationship between time-steps by sharing same parameters among all time-steps. Given this issue, we propose tensor recurrent neural network with Gaussian mixture model (GmTRNN) for HRRP, not only making use of temporal characteristic but also modeling the variation among its patterns. Specifically, a novel tensor RNN is developed by extending all the parameters in the form of tensor to explore diverse temporal dependence between range cells within an HRRP sample, where a mixture model is introduced to determine the parameters of each time-step in tensor RNN. Moreover, to take advantage of Bayesian nonparametrics in handling the unknown number of mixture components, we further propose the tensor recurrent neural network with Dirichlet process mixture (DPmTRNN). For scalable and joint training of clustering and recognition, we present effective hybrid online variational inference and stochastic gradient descent method. Experiments on benchmark data, measured and simulated HRRP data demonstrate the the effectiveness and efficiency of our models and its robustness to HRRP shift.

Nonparametric Hierarchical Hidden Semi-Markov Model for Brain Fatigue Behavior Detection of Pilots During Flight

Abstract: The evaluation of pilot brain activity is very important for flight safety. This study proposes a Hidden semi-Markov Model with Hierarchical prior to detect brain activity under different flight tasks. A dynamic student mixture model is proposed to detect the outlier of emission probability of HSMM. Instantaneous spectrum features are also extracted from EEG signals. Compared with other latent variable models, the proposed model shows excellent performance for the automatic inference of brain cognitive activity of pilots. The results indicate that the consideration of hierarchical model and the emission probability with t mixture model improves the recognition performance for Pilots' fatigue cognitive level.