High-dimensional and large-scale anomaly detection using a linear one-class SVM with deep learning

https://sci2s.ugr.es/sites/default/files/ficherosPublicaciones/1772_bol14_INS_FS_microarray.pdf

A review of microarray datasets and applied feature selection methods

Statistical models of text have become increasingly popular in statistics and computer science as a method of exploring large document collections. Social scientists often want to move beyond exploration, to measurement and experimentation, and make inference about social and political processes that drive discourse and content. In this article, we develop a model of text data that supports this type of substantive research. Our approach is to posit a hierarchical mixed membership model for analyzing topical content of documents, in which mixing weights are parameterized by observed covariates. In this model, topical prevalence and topical content are specified as a simple generalized linear model on an arbitrary number of document-level covariates, such as news source and time of release, enabling researchers to introduce elements of the experimental design that informed document collection into the model, within a generally applicable framework. We demonstrate the proposed methodology by analyzi...

A model of text for experimentation in the social sciences

This work studies the Generalized Singular Value Thresholding (GSVT) operator Proxσg(·), Proxσg(B) = arg minx ∑mi=1 g(σi(X)) + 1/2 ||X - B||2F, associated with a nonconvex function g defined on the singular values of X. We prove that GSVT can be obtained by performing the proximal operator of g (denoted as Proxg(·)) on the singular values since Proxg(·) is monotone when g is lower bounded. If the nonconvex g satisfies some conditions (many popular nonconvex surrogate functions, e.g., lp-norm, 0 < p < 1, of l0-norm are special cases), a general solver to find Proxg(b) is proposed for any b ≥ 0. GSVT greatly generalizes the known Singular Value Thresholding (SVT) which is a basic subroutine in many convex low rank minimization methods. We are able to solve the nonconvex low rank minimization problem by using GSVT in place of SVT.

Generalized Singular Value Thresholding

Graphical abstractDisplay Omitted HighlightsWe propose a novel binary coded ant colony algorithm by blending of GA and BACO.The proposed algorithm is adopted to handle with the problem of feature selection.Results show the method outperforms GA, BPSO, BACO, ABACO, BDE in feature selection. Feature selection is a significant task for data mining and pattern recognition. It aims to select the optimal feature subset with the minimum redundancy and the maximum discriminating ability. In the paper, a feature selection approach based on a modified binary coded ant colony optimization algorithm (MBACO) combined with genetic algorithm (GA) is proposed. The method comprises two models, which are the visibility density model (VMBACO) and the pheromone density model (PMBACO). In VMBACO, the solution obtained by GA is used as visibility information; on the other hand, in PMBACO, the solution obtained by GA is used as initial pheromone information. In the method, each feature is treated as a binary bit and each bit has two orientations, one is for selecting the feature and another is for deselecting. The proposed method is also compared with that of GA, binary coded ant colony optimization (BACO), advanced BACO (ABACO), binary coded particle swarm optimization (BPSO), binary coded differential evolution (BDE) and a hybrid GA-ACO algorithm on some well-known UCI datasets; furthermore, it is also compared with some other existing techniques such as minimum Redundancy Maximum Relevance (mRMR), Relief algorithm for a comprehensive comparison. Experimental results display that the proposed method is robust, adaptive and exhibits the better performance than other methods involved in the paper.

A feature selection method based on modified binary coded ant colony optimization algorithm

Motivated by the recent developments of nonconvex penalties in sparsity modeling, we propose a nonconvex optimization model for handing the low-rank matrix recovery problem. Different from the famous robust principal component analysis (RPCA), we suggest recovering low-rank and sparse matrices via a nonconvex loss function and a nonconvex penalty. The advantage of the nonconvex approach lies in its stronger robustness. To solve the model, we devise a majorization-minimization augmented Lagrange multiplier (MM-ALM) algorithm which finds the local optimal solutions of the proposed nonconvex model. We also provide an efficient strategy to speedup MM-ALM, which makes the running time comparable with the state-of-the-art algorithm of solving RPCA. Finally, empirical results demonstrate the superiority of our nonconvex approach over RPCA in terms of matrix recovery accuracy.

Nonconvex relaxation approaches to robust matrix recovery

In this paper we propose a novel framework for the construction of sparsity-inducing priors. In particular, we define such priors as a mixture of exponential power distributions with a generalized inverse Gaussian density (EP-GIG). EP-GIG is a variant of generalized hyperbolic distributions, and the special cases include Gaussian scale mixtures and Laplace scale mixtures. Furthermore, Laplace scale mixtures can subserve a Bayesian framework for sparse learning with nonconvex penalization. The densities of EP-GIG can be explicitly expressed. Moreover, the corresponding posterior distribution also follows a generalized inverse Gaussian distribution. We exploit these properties to develop EM algorithms for sparse empirical Bayesian learning. We also show that these algorithms bear an interesting resemblance to iteratively reweighted l2 or l1 methods. Finally, we present two extensions for grouped variable selection and logistic regression.

/pdf/ep-gig-priors-and-applications-in-bayesian-sparse-learning-52iier5mcp.pdf

EP-GIG priors and applications in bayesian sparse learning

An iterative SVM approach to feature selection and classification in high-dimensional datasets

Low-rank matrix completion is an important theme both theoretically and practically. However, the state-of-the-art methods based on convex optimization usually lead to a certain amount of deviation from the original matrix. To perfectly recover a data matrix from a sampling of its entries, we consider a non-convex alternative to approximate the matrix rank. In particular, we minimize a matrix γ-norm under a set of linear constraints. Accordingly, we derive a shrinkage operator, which is nearly unbiased in comparison with the well-known soft shrinkage operator. Furthermore, we devise two algorithms, non-convex soft imputation (NCSI) and non-convex alternative direction method of multipliers (NCADMM), to fulfil the numerical estimation. Experimental results show that these algorithms outperform existing matrix completion methods in accuracy. Moreover, the NCADMM is as efficient as the current state-of-the-art algorithms.

/pdf/a-nearly-unbiased-matrix-completion-approach-5goid49rsv.pdf

A Nearly Unbiased Matrix Completion Approach

In this paper we propose a novel framework for the construction of sparsity-inducing priors. In particular, we define such priors as a mixture of exponential power distributions with a generalized inverse Gaussian density (EP-GIG). EP-GIG is a variant of generalized hyperbolic distributions, and the special cases include Gaussian scale mixtures and Laplace scale mixtures. Furthermore, Laplace scale mixtures can subserve a Bayesian framework for sparse learning with nonconvex penalization. The densities of EP-GIG can be explicitly expressed. Moreover, the corresponding posterior distribution also follows a generalized inverse Gaussian distribution. These properties lead us to EM algorithms for Bayesian sparse learning. We show that these algorithms bear an interesting resemblance to iteratively re-weighted $\ell_2$ or $\ell_1$ methods. In addition, we present two extensions for grouped variable selection and logistic regression.

Dehua Liu

Papers

Nonconvex relaxation approaches to robust matrix recovery

EP-GIG priors and applications in bayesian sparse learning

An iterative SVM approach to feature selection and classification in high-dimensional datasets

A Nearly Unbiased Matrix Completion Approach

EP-GIG Priors and Applications in Bayesian Sparse Learning