Showing papers on "Outlier published in 2022"

Ensemble unsupervised autoencoders and Gaussian mixture model for cyberattack detection

Abstract: Previous studies have adopted unsupervised machine learning with dimension reduction functions for cyberattack detection, which are limited to performing robust anomaly detection with high-dimensional and sparse data. Most of them usually assume homogeneous parameters with a specific Gaussian distribution for each domain, ignoring the robust testing of data skewness. This paper proposes to use unsupervised ensemble autoencoders connected to the Gaussian mixture model (GMM) to adapt to multiple domains regardless of the skewness of each domain. In the hidden space of the ensemble autoencoder, the attention-based latent representation and reconstructed features of the minimum error are utilized. The expectation maximization (EM) algorithm is used to estimate the sample density in the GMM. When the estimated sample density exceeds the learning threshold obtained in the training phase, the sample is identified as an outlier related to an attack anomaly. Finally, the ensemble autoencoder and the GMM are jointly optimized, which transforms the optimization of objective function into a Lagrangian dual problem. Experiments conducted on three public data sets validate that the performance of the proposed model is significantly competitive with the selected anomaly detection baselines. • An ensemble framework of multichannel network anomaly detection model that combines deep autoencoders and the GMM. • A robust optimization version of EM 3 for multiple domains, which transforms the optimization problem of the objective function into a Lagrangian dual. • We deduce the formula and analyze the convergence of the full text, and prove that our model has stability and robustness. • To the best of our knowledge is the first work that performs algorithms on both differentiated data domains and data distributions.

Deep-Learning-Based Open Set Fault Diagnosis by Extreme Value Theory

Abstract: Existing data-driven fault diagnosis methods assume that the label sets of the training data and test data are consistent, which is usually not applicable for real applications since the fault modes that occur in the test phase are unpredictable. To address this problem, open set fault diagnosis (OSFD), where the test label set consists of a portion of the training label set and some unknown classes, is studied in this article. Considering the changeable operating conditions of machinery, OSFD tasks are further divided into shared-domain open set fault diagnosis (SOSFD) and cross-domain open set fault diagnosis (COSFD) in this article. For SOSFD, 1-D convolutional neural networks are trained for learning discriminative features and recognizing fault modes. For COSFD, due to the distribution discrepancy between the source and target domains, the deep model needs to learn domain-invariant features of shared classes and separate features of outlier classes. Thus, by utilizing the output of an additional domain classifier, a model named bilateral weighted adversarial networks is proposed to assign large weights to shared classes and small weights to outlier classes during the feature alignment. In the test phase, samples are classified according to the outputs of the deep model and unknown-class samples are rejected by the extreme value theory model. Experimental results on two bearing datasets demonstrate the effectiveness and superiority of the proposed method.

Multiview Learning With Robust Double-Sided Twin SVM

Abstract: Multiview learning (MVL), which enhances the learners’ performance by coordinating complementarity and consistency among different views, has attracted much attention. The multiview generalized eigenvalue proximal support vector machine (MvGSVM) is a recently proposed effective binary classification method, which introduces the concept of MVL into the classical generalized eigenvalue proximal support vector machine (GEPSVM). However, this approach cannot guarantee good classification performance and robustness yet. In this article, we develop multiview robust double-sided twin SVM (MvRDTSVM) with SVM-type problems, which introduces a set of double-sided constraints into the proposed model to promote classification performance. To improve the robustness of MvRDTSVM against outliers, we take L1-norm as the distance metric. Also, a fast version of MvRDTSVM (called MvFRDTSVM) is further presented. The reformulated problems are complex, and solving them are very challenging. As one of the main contributions of this article, we design two effective iterative algorithms to optimize the proposed nonconvex problems and then conduct theoretical analysis on the algorithms. The experimental results verify the effectiveness of our proposed methods.

An <i>L</i> ₁-and-<i>L</i> ₂-Norm-Oriented Latent Factor Model for Recommender Systems

Abstract: A recommender system (RS) is highly efficient in filtering people's desired information from high-dimensional and sparse (HiDS) data. To date, a latent factor (LF)-based approach becomes highly popular when implementing a RS. However, current LF models mostly adopt single distance-oriented Loss like an L2 norm-oriented one, which ignores target data's characteristics described by other metrics like an L1 norm-oriented one. To investigate this issue, this article proposes an L1 -and- L2 -norm-oriented LF ( [Formula: see text]) model. It adopts twofold ideas: 1) aggregating L1 norm's robustness and L2 norm's stability to form its Loss and 2) adaptively adjusting weights of L1 and L2 norms in its Loss. By doing so, it achieves fine aggregation effects with L1 norm-oriented Loss 's robustness and L2 norm-oriented Loss 's stability to precisely describe HiDS data with outliers. Experimental results on nine HiDS datasets generated by real systems show that an [Formula: see text] model significantly outperforms state-of-the-art models in prediction accuracy for missing data of an HiDS dataset. Its computational efficiency is also comparable with the most efficient LF models. Hence, it has good potential for addressing HiDS data from real applications.

Towards Total Recall in Industrial Anomaly Detection

Abstract: Being able to spot defective parts is a critical component in large-scale industrial manufacturing. A particular challenge that we address in this work is the cold-start problem: fit a model using nominal (non-defective) example images only. While handcrafted solutions per class are possible, the goal is to build systems that work well simultaneously on many different tasks automatically. The best peforming approaches combine embeddings from ImageNet models with an outlier detection model. In this paper, we extend on this line of work and propose PatchCore, which uses a maximally representative memory bank of nominal patch-features. PatchCore offers competitive inference times while achieving state-of-the-art performance for both detection and localization. On the challenging, widely used MVTec AD benchmark PatchCore achieves an image-level anomaly detection AUROC score of up to 99.6%, more than halving the error compared to the next best competitor. We further report competitive results on two additional datasets and also find competitive results in the few samples regime. Code: github.com/amazon-research/patchcore-inspection.

Ultimately Bounded Filtering Subject to Impulsive Measurement Outliers

Abstract: This article is concerned with the ultimately bounded filtering problem for a class of linear time-delay systems subject to norm-bounded disturbances and impulsive measurement outliers (IMOs). The considered IMOs are modeled by a sequence of impulsive signals with certain known minimum norm (i.e., the minimum of the norms of all impulsive signals). In order to characterize the occasional occurrence of IMOs, a sequence of independent and identically distributed random variables is introduced to depict the interval lengths (i.e., the durations between two adjacent IMOs) of the outliers. In order to achieve satisfactory filtering performance, a novel parameter-dependent filtering approach is proposed to protect the filtering performance from IMOs by using a special outlier detection scheme, which is developed based on a particular input–output model. First, the ultimate boundedness (in mean square) of the filtering error is investigated by using the stochastic analysis technique and the Lyapunov-functional-like method. Then, the desired filter gain matrix is derived through solving a constrained optimization problem. Furthermore, the designed filtering scheme is applied to the case where the statistical properties about the interval lengths of outliers are completely unknown. Finally, a simulation example is provided to demonstrate the effectiveness of our proposed filtering strategy.

Learning Robust Discriminant Subspace Based on Joint L₂,<i>ₚ</i>- and L₂,<i>ₛ</i>-Norm Distance Metrics

Abstract: Recently, there are many works on discriminant analysis, which promote the robustness of models against outliers by using L1- or L2,1-norm as the distance metric. However, both of their robustness and discriminant power are limited. In this article, we present a new robust discriminant subspace (RDS) learning method for feature extraction, with an objective function formulated in a different form. To guarantee the subspace to be robust and discriminative, we measure the within-class distances based on [Formula: see text]-norm and use [Formula: see text]-norm to measure the between-class distances. This also makes our method include rotational invariance. Since the proposed model involves both [Formula: see text]-norm maximization and [Formula: see text]-norm minimization, it is very challenging to solve. To address this problem, we present an efficient nongreedy iterative algorithm. Besides, motivated by trace ratio criterion, a mechanism of automatically balancing the contributions of different terms in our objective is found. RDS is very flexible, as it can be extended to other existing feature extraction techniques. An in-depth theoretical analysis of the algorithm's convergence is presented in this article. Experiments are conducted on several typical databases for image classification, and the promising results indicate the effectiveness of RDS.

Oriented RepPoints for Aerial Object Detection

Abstract: In contrast to the generic object, aerial targets are often non-axis aligned with arbitrary orientations having the cluttered surroundings. Unlike the mainstreamed approaches regressing the bounding box orientations, this paper proposes an effective adaptive points learning approach to aerial object detection by taking advantage of the adaptive points representation, which is able to capture the geometric information of the arbitrary-oriented instances. To this end, three oriented conversion functions are presented to facilitate the classification and localization with accurate orientation. Moreover, we propose an effective quality assessment and sample assignment scheme for adaptive points learning toward choosing the representative oriented reppoints samples during training, which is able to capture the non-axis aligned features from adjacent objects or background noises. A spatial constraint is introduced to penalize the outlier points for roust adaptive learning. Experimental results on four challenging aerial datasets including DOTA, HRSC2016, UCAS-AOD and DIOR-R, demonstrate the efficacy of our proposed approach. The source code is availabel at: https://github.com/LiWentomng/OrientedRepPoints.

Self-Paced Dynamic Infinite Mixture Model for Fatigue Evaluation of Pilots’ Brains

Abstract: Current brain cognitive models are insufficient in handling outliers and dynamics of electroencephalogram (EEG) signals. This article presents a novel self-paced dynamic infinite mixture model to infer the dynamics of EEG fatigue signals. The instantaneous spectrum features provided by ensemble wavelet transform and Hilbert transform are extracted to form four fatigue indicators. The covariance of log likelihood of the complete data is proposed to accurately identify similar components and dynamics of the developed mixture model. Compared with its seven peers, the proposed model shows better performance in automatically identifying a pilot's brain workload.

Deep-Learning-Based Open Set Fault Diagnosis by Extreme Value Theory

Abstract: Existing data-driven fault diagnosis methods assume that the label sets of the training data and test data are consistent, which is usually not applicable for real applications since the fault modes that occur in the test phase are unpredictable. To address this problem, open set fault diagnosis (OSFD), where the test label set consists of a portion of the training label set and some unknown classes, is studied in this article. Considering the changeable operating conditions of machinery, OSFD tasks are further divided into shared-domain open set fault diagnosis (SOSFD) and cross-domain open set fault diagnosis (COSFD) in this article. For SOSFD, 1-D convolutional neural networks are trained for learning discriminative features and recognizing fault modes. For COSFD, due to the distribution discrepancy between the source and target domains, the deep model needs to learn domain-invariant features of shared classes and separate features of outlier classes. Thus, by utilizing the output of an additional domain classifier, a model named bilateral weighted adversarial networks is proposed to assign large weights to shared classes and small weights to outlier classes during the feature alignment. In the test phase, samples are classified according to the outputs of the deep model and unknown-class samples are rejected by the extreme value theory model. Experimental results on two bearing datasets demonstrate the effectiveness and superiority of the proposed method.

An Adaptive Kalman Filter With Inaccurate Noise Covariances in the Presence of Outliers

Abstract: In this article, a novel variational Bayesian (VB) adaptive Kalman filter with inaccurate nominal process and measurement noise covariances (PMNC) in the presence of outliers is proposed. The probability density functions of state transition and measurement likelihood are modeled as Gaussian–Gamma mixture distributions. The VB inference is used to perform the state and PMNC simultaneously. Simulations show that the effectiveness of the proposed method with inaccurate noise covariances in the presence of outliers environments.

Meta-analysis fine-mapping is often miscalibrated at single-variant resolution

Abstract: Meta-analysis is pervasively used to combine multiple genome-wide association studies (GWASs). Fine-mapping of meta-analysis studies is typically performed as in a single-cohort study. Here, we first demonstrate that heterogeneity (e.g., of sample size, phenotyping, imputation) hurts calibration of meta-analysis fine-mapping. We propose a summary statistics-based quality-control (QC) method, suspicious loci analysis of meta-analysis summary statistics (SLALOM), that identifies suspicious loci for meta-analysis fine-mapping by detecting outliers in association statistics. We validate SLALOM in simulations and the GWAS Catalog. Applying SLALOM to 14 meta-analyses from the Global Biobank Meta-analysis Initiative (GBMI), we find that 67% of loci show suspicious patterns that call into question fine-mapping accuracy. These predicted suspicious loci are significantly depleted for having nonsynonymous variants as lead variant (2.7×; Fisher's exact p = 7.3 × 10-4). We find limited evidence of fine-mapping improvement in the GBMI meta-analyses compared with individual biobanks. We urge extreme caution when interpreting fine-mapping results from meta-analysis of heterogeneous cohorts.

Artificial Intelligence Enhanced Two-Stage Hybrid Fault Prognosis Methodology of PMSM

Abstract: Fault prognosis based on single model is generally inaccurate due to the varying working conditions. A multistage fault prognosis methodology combining stage identification with Bayesian networks (BNs) and time series approach with particular emphasis on the autoregressive moving average (ARMA) model is proposed to solve this problem. In the first stage, degradation data are identified, and outliers are marked by the Euclidean distance. Degenerate attributes of outliers are finely identified by BNs and matched to the corresponding model. In the second stage, the ARMA model is used for prognosis according to the results of the fine identification. Subsequently, the double-precision identification and ARMA submodel prognosis are carried out alternately throughout the prognosis process. Three degradation types of permanent magnet synchronous motor are simulated to verify the applicability of the method. Result shows that it can track the changes in the degradation in time and obtains better results.

Alzheimer's Disease Diagnosis via Intuitionistic Fuzzy Random Vector Functional Link Network

Abstract: Alzheimer's disease (AD) is a prominent neurodegenerative disorder, which leads to memory loss and cognitive impairment. The progression is irreversible and shows atrophies in cerebral cortex. Multiple studies revealed that the early diagnosis and early treatment can slow the progress of dementia, and hence, further atrophies can be controlled. Brain imaging data, such as magnetic resonance imaging (MRI), have been prominently used for the diagnosis of AD. Multiple approaches have been proposed for the diagnosis of AD. We propose a novel intuitionistic fuzzy random vector functional link network (IFRVFL) for the diagnosis of AD. Unlike standard random vector functional link (RVFL) network, extreme learning machine (ELM), and kernel ridge regression (KRR), which uses a uniform weighting approach for generating the optimal classifier, the proposed IFRVFL uses a fuzzy weighted approach for generating the optimal classifier. A uniform weighting scheme assumes that all the data samples are equally important; however, in real-world scenarios, this assumption may not hold true due to the presence of outliers and noise. Hence, it results in lower generalization. The proposed IFRVFL assigns each sample an intuitionistic fuzzy number (IFN), which is a function of membership and nonmembership score of a sample. The membership score is a function of the sample distance from the centroid of its corresponding class and the nonmembership score is a function of sample distance from the centroid as well as the neighborhood of the given sample. The proposed IFRVFL effectively reduces the influence of outliers. To evaluate the efficiency of the proposed IFRVFL model, we employed it for the diagnosis of AD. Experimental results demonstrate that the proposed IFRVFL model is superior in mild cognitive impairment (MCI) versus AD case. Thus, IFRVFL can be used in the clinical setting for the early diagnosis of AD. Furthermore, to check the robustness of the proposed IFRVFL model, we also evaluated it on benchmark datasets. Experimental results and the statistical tests reveal that the proposed IFRVFL is better in comparison to baseline models. The source code of the proposed IFRVFL formulation is available at https://github.com/mtanveer1/.

A review of machine learning methods for drought hazard monitoring and forecasting: Current research trends, challenges, and future research directions

Abstract: Machine learning is a dynamic field with wide-ranging applications, including drought modeling and forecasting. Drought is a complex, devastating natural disaster for which it is challenging to develop effective prediction models. Therefore, our review focuses on basic information about machine learning methods (MLMs) and their potential applications in developing efficient and effective drought forecasting models. We observed that MLMs have achieved significant advances in the robustness, effectiveness, and accuracy of the algorithms for drought modelling in recent years. The performance comparison of MLMs with other models provides a comprehensive conception of different model evaluation metrics. Further challenges of MLMs, such as inadequate training data sets, noise, outliers, and observation bias for spatial data sets, are explored. Finally, our review conveys in-depth understanding to researchers on machine learning applications in forecasting and modeling and provides drought mitigation strategy guidance for policymakers.

A Survey of Outlier Detection Techniques in IoT: Review and Classification

Abstract: The Internet of Things (IoT) is a fact today where a high number of nodes are used for various applications. From small home networks to large-scale networks, the aim is the same: transmitting data from the sensors to the base station. However, these data are susceptible to different factors that may affect the collected data efficiency or the network functioning, and therefore the desired quality of service (QoS). In this context, one of the main issues requiring more research and adapted solutions is the outlier detection problem. The challenge is to detect outliers and classify them as either errors to be ignored, or important events requiring actions to prevent further service degradation. In this paper, we propose a comprehensive literature review of recent outlier detection techniques used in the IoTs context. First, we provide the fundamentals of outlier detection while discussing the different sources of an outlier, the existing approaches, how we can evaluate an outlier detection technique, and the challenges facing designing such techniques. Second, comparison and discussion of the most recent outlier detection techniques are presented and classified into seven main categories, which are: statistical-based, clustering-based, nearest neighbour-based, classification-based, artificial intelligent-based, spectral decomposition-based, and hybrid-based. For each category, available techniques are discussed, while highlighting the advantages and disadvantages of each of them. The related works for each of them are presented. Finally, a comparative study for these techniques is provided.

Self-Distilled StyleGAN: Towards Generation from Internet Photos

Abstract: StyleGAN is known to produce high-fidelity images, while also offering unprecedented semantic editing. However, these fascinating abilities have been demonstrated only on a limited set of datasets, which are usually structurally aligned and well curated. In this paper, we show how StyleGAN can be adapted to work on raw uncurated images collected from the Internet. Such image collections impose two main challenges to StyleGAN: they contain many outlier images, and are characterized by a multi-modal distribution. Training StyleGAN on such raw image collections results in degraded image synthesis quality. To meet these challenges, we proposed a StyleGAN-based self-distillation approach, which consists of two main components: (i) A generative-based self-filtering of the dataset to eliminate outlier images, in order to generate an adequate training set, and (ii) Perceptual clustering of the generated images to detect the inherent data modalities, which are then employed to improve StyleGAN’s “truncation trick” in the image synthesis process. The presented technique enables the generation of high-quality images, while minimizing the loss in diversity of the data. Through qualitative and quantitative evaluation, we demonstrate the power of our approach to new challenging and diverse domains collected from the Internet. New datasets and pre-trained models are provided in our project website https://self-distilled-stylegan.github.io/.