Showing papers by "Nathan S. Netanyahu published in 2017"

DeepSign: Deep Learning for Automatic Malware Signature Generation and Classification

Abstract: This paper presents a novel deep learning based method for automatic malware signature generation and classification. The method uses a deep belief network (DBN), implemented with a deep stack of denoising autoencoders, generating an invariant compact representation of the malware behavior. While conventional signature and token based methods for malware detection do not detect a majority of new variants for existing malware, the results presented in this paper show that signatures generated by the DBN allow for an accurate classification of new malware variants. Using a dataset containing hundreds of variants for several major malware families, our method achieves 98.6% classification accuracy using the signatures generated by the DBN. The presented method is completely agnostic to the type of malware behavior that is logged (e.g., API calls and their parameters, registry entries, websites and ports accessed, etc.), and can use any raw input from a sandbox to successfully train the deep neural network which is used to generate malware signatures.

A Stepwise Analytical Projected Gradient Descent Search for Hyperspectral Unmixing and Its Code Vectorization

Abstract: We present, in this paper, a new methodology for spectral unmixing, where a vector of fractions, corresponding to a set of endmembers (EMs), is estimated for each pixel in the image. The process first provides an initial estimate of the fraction vector, followed by an iterative procedure that converges to an optimal solution. Specifically, projected gradient descent (PGD) optimization is applied to (a variant of) the spectral angle mapper objective function, so as to significantly reduce the estimation error due to amplitude (i.e., magnitude) variations in EM spectra, caused by the illumination change effect. To improve the computational efficiency of our method over a commonly used gradient descent technique, we have analytically derived the objective function’s gradient and the optimal step size (used in each iteration). To gain further improvement, we have implemented our unmixing module via code vectorization, where the entire process is “folded” into a single loop, and the fractions for all of the pixels are solved simultaneously. We call this new parallel scheme vectorized code PGD unmixing (VPGDU). VPGDU has the advantage of solving (simultaneously) an independent optimization problem per image pixel, exactly as other pixelwise algorithms, but significantly faster. Its performance was compared with the commonly used fully constrained least squares unmixing (FCLSU), the generalized bilinear model (GBM) method for hyperspectral unmixng, and the fast state-of-the-art methods, sparse unmixing by variable splitting and augmented Lagrangian (SUnSAL) and collaborative SUnSAL (CLSUnSAL) based on the alternating direction method of multipliers. Considering all of the prospective EMs of a scene at each pixel (i.e., without a priori knowledge which/how many EMs are actually present in a given pixel), we demonstrate that the accuracy due to VPGDU is considerably higher than that obtained by FCLSU, GBM, SUnSAL, and CLSUnSAL under varying illumination, and is, otherwise, comparable with respect to these methods. However, while our method is significantly faster than FCLSU and GBM, it is slower than SUnSAL and CLSUnSAL by roughly an order of magnitude.

A Genetic Algorithm-Based Solver for Very Large Jigsaw Puzzles

Abstract: In this paper we propose the first effective automated, genetic algorithm (GA)-based jigsaw puzzle solver. We introduce a novel procedure of merging two "parent" solutions to an improved "child" solution by detecting, extracting, and combining correctly assembled puzzle segments. The solver proposed exhibits state-of-the-art performance solving previously attempted puzzles faster and far more accurately, and also puzzles of size never before attempted. Other contributions include the creation of a benchmark of large images, previously unavailable. We share the data sets and all of our results for future testing and comparative evaluation of jigsaw puzzle solvers.

A Generalized Genetic Algorithm-Based Solver for Very Large Jigsaw Puzzles of Complex Types

Abstract: In this paper we introduce new types of square-piece jigsaw puzzles, where in addition to the unknown location and orientation of each piece, a piece might also need to be flipped. These puzzles, which are associated with a number of real world problems, are considerably harder, from a computational standpoint. Specifically, we present a novel generalized genetic algorithm (GA)-based solver that can handle puzzle pieces of unknown location and orientation (Type 2 puzzles) and (two-sided) puzzle pieces of unknown location, orientation, and face (Type 4 puzzles). To the best of our knowledge, our solver provides a new state-of-the-art, solving previously attempted puzzles faster and far more accurately, handling puzzle sizes that have never been attempted before, and assembling the newly introduced two-sided puzzles automatically and effectively. This paper also presents, among other results, the most extensive set of experimental results, compiled as of yet, on Type 2 puzzles.

Genetic Algorithms for Evolving Computer Chess Programs

Abstract: This paper demonstrates the use of genetic algorithms for evolving: 1) a grandmaster-level evaluation function, and 2) a search mechanism for a chess program, the parameter values of which are initialized randomly. The evaluation function of the program is evolved by learning from databases of (human) grandmaster games. At first, the organisms are evolved to mimic the behavior of human grandmasters, and then these organisms are further improved upon by means of coevolution. The search mechanism is evolved by learning from tactical test suites. Our results show that the evolved program outperforms a two-time world computer chess champion and is at par with the other leading computer chess programs.

DeepChess: End-to-End Deep Neural Network for Automatic Learning in Chess

Abstract: We present an end-to-end learning method for chess, relying on deep neural networks. Without any a priori knowledge, in particular without any knowledge regarding the rules of chess, a deep neural network is trained using a combination of unsupervised pretraining and supervised training. The unsupervised training extracts high level features from a given position, and the supervised training learns to compare two chess positions and select the more favorable one. The training relies entirely on datasets of several million chess games, and no further domain specific knowledge is incorporated. The experiments show that the resulting neural network (referred to as DeepChess) is on a par with state-of-the-art chess playing programs, which have been developed through many years of manual feature selection and tuning. DeepChess is the first end-to-end machine learning-based method that results in a grandmaster-level chess playing performance.

An Automatic Solver for Very Large Jigsaw Puzzles Using Genetic Algorithms

Abstract: In this paper we propose the first effective genetic algorithm (GA)-based jigsaw puzzle solver. We introduce a novel crossover procedure that merges two "parent" solutions to an improved "child" configuration by detecting, extracting, and combining correctly assembled puzzle segments. The solver proposed exhibits state-of-the-art performance, as far as handling previously attempted puzzles more accurately and efficiently, as well puzzle sizes that have not been attempted before. The extended experimental results provided in this paper include, among others, a thorough inspection of up to 30,745-piece puzzles (compared to previous attempts on 22,755-piece puzzles), using a considerably faster concurrent implementation of the algorithm. Furthermore, we explore the impact of different phases of the novel crossover operator by experimenting with several variants of the GA. Finally, we compare different fitness functions and their effect on the overall results of the GA-based solver.

Genetic Algorithm-Based Solver for Very Large Multiple Jigsaw Puzzles of Unknown Dimensions and Piece Orientation

Abstract: In this paper we propose the first genetic algorithm (GA)-based solver for jigsaw puzzles of unknown puzzle dimensions and unknown piece location and orientation. Our solver uses a novel crossover technique, and sets a new state-of-the-art in terms of the puzzle sizes solved and the accuracy obtained. The results are significantly improved, even when compared to previous solvers assuming known puzzle dimensions. Moreover, the solver successfully contends with a mixed bag of multiple puzzle pieces, assembling simultaneously all puzzles.

DeepPainter: Painter Classification Using Deep Convolutional Autoencoders

Abstract: In this paper we describe the problem of painter classification, and propose a novel approach based on deep convolutional autoencoder neural networks. While previous approaches relied on image processing and manual feature extraction from paintings, our approach operates on the raw pixel level, without any preprocessing or manual feature extraction. We first train a deep convolutional autoencoder on a dataset of paintings, and subsequently use it to initialize a supervised convolutional neural network for the classification phase. The proposed approach substantially outperforms previous methods, improving the previous state-of-the-art for the 3-painter classification problem from 90.44% accuracy (previous state-of-the-art) to 96.52% accuracy, i.e., a 63% reduction in error rate.

DNN-Buddies: A Deep Neural Network-Based Estimation Metric for the Jigsaw Puzzle Problem

Abstract: This paper introduces the first deep neural network-based estimation metric for the jigsaw puzzle problem. Given two puzzle piece edges, the neural network predicts whether or not they should be adjacent in the correct assembly of the puzzle, using nothing but the pixels of each piece. The proposed metric exhibits an extremely high precision even though no manual feature extraction is performed. When incorporated into an existing puzzle solver, the solution's accuracy increases significantly, achieving thereby a new state-of-the-art standard.

DeepBrain: Functional Representation of Neural In-Situ Hybridization Images for Gene Ontology Classification Using Deep Convolutional Autoencoders

Abstract: This paper presents a novel deep learning-based method for learning a functional representation of mammalian neural images. The method uses a deep convolutional denoising autoencoder (CDAE) for generating an invariant, compact representation of in situ hybridization (ISH) images. While most existing methods for bio-imaging analysis were not developed to handle images with highly complex anatomical structures, the results presented in this paper show that functional representation extracted by CDAE can help learn features of functional gene ontology categories for their classification in a highly accurate manner. Using this CDAE representation, our method outperforms the previous state-of-the-art classification rate, by improving the average AUC from 0.92 to 0.98, i.e., achieving 75% reduction in error. The method operates on input images that were downsampled significantly with respect to the original ones to make it computationally feasible.

A two-phase genetic algorithm for image registration

Abstract: Image Registration (IR) is the process of aligning two (or more) images of the same scene taken at different times, different viewpoints and/or by different sensors. It is an important, crucial step in various image analysis tasks where multiple data sources are integrated/fused, in order to extract high-level information. Registration methods usually assume a relevant transformation model for a given problem domain. The goal is to search for the "optimal" instance of the transformation model assumed with respect to a similarity measure in question. In this paper we present a novel genetic algorithm (GA)-based approach for IR. Since GA performs effective search in various optimization problems, it could prove useful also for IR. Indeed, various GAs have been proposed for IR. However, most of them assume certain constraints, which simplify the transformation model, restrict the search space or make additional preprocessing requirements. In contrast, we present a generalized GA-based solution for an almost fully affine transformation model, which achieves competitive results without such limitations using a two-phase method and a multi-objective optimization (MOO) approach. We present good results for multiple dataset and demonstrate the robustness of our method in the presence of noisy data.

Expert-Driven Genetic Algorithms for Simulating Evaluation Functions

Abstract: In this paper we demonstrate how genetic algorithms can be used to reverse engineer an evaluation function's parameters for computer chess. Our results show that using an appropriate expert (or mentor), we can evolve a program that is on par with top tournament-playing chess programs, outperforming a two-time World Computer Chess Champion. This performance gain is achieved by evolving a program that mimics the behavior of a superior expert. The resulting evaluation function of the evolved program consists of a much smaller number of parameters than the expert's. The extended experimental results provided in this paper include a report of our successful participation in the 2008 World Computer Chess Championship. In principle, our expert-driven approach could be used in a wide range of problems for which appropriate experts are available.

DeepBrain: Functional Representation of Neural In-Situ Hybridization Images for Gene Ontology Classification Using Deep Convolutional Autoencoders

Abstract: This paper presents a novel deep learning-based method for learning a functional representation of mammalian neural images. The method uses a deep convolutional denoising autoencoder (CDAE) for generating an invariant, compact representation of in situ hybridization (ISH) images. While most existing methods for bio-imaging analysis were not developed to handle images with highly complex anatomical structures, the results presented in this paper show that functional representation extracted by CDAE can help learn features of functional gene ontology categories for their classification in a highly accurate manner. Using this CDAE representation, our method outperforms the previous state-of-the-art classification rate, by improving the average AUC from 0.92 to 0.98, i.e., achieving 75% reduction in error. The method operates on input images that were downsampled significantly with respect to the original ones to make it computationally feasible.

A Two-Phase Genetic Algorithm for Image Registration

Abstract: Image Registration (IR) is the process of aligning two (or more) images of the same scene taken at different times, different viewpoints and/or by different sensors. It is an important, crucial step in various image analysis tasks where multiple data sources are integrated/fused, in order to extract high-level information. Registration methods usually assume a relevant transformation model for a given problem domain. The goal is to search for the "optimal" instance of the transformation model assumed with respect to a similarity measure in question. In this paper we present a novel genetic algorithm (GA)-based approach for IR. Since GA performs effective search in various optimization problems, it could prove useful also for IR. Indeed, various GAs have been proposed for IR. However, most of them assume certain constraints, which simplify the transformation model, restrict the search space or make additional preprocessing requirements. In contrast, we present a generalized GA-based solution for an almost fully affine transformation model, which achieves competitive results without such limitations using a two-phase method and a multi-objective optimization (MOO) approach. We present good results for multiple dataset and demonstrate the robustness of our method in the presence of noisy data.

Simulating Human Grandmasters: Evolution and Coevolution of Evaluation Functions

Abstract: This paper demonstrates the use of genetic algorithms for evolving a grandmaster-level evaluation function for a chess program. This is achieved by combining supervised and unsupervised learning. In the supervised learning phase the organisms are evolved to mimic the behavior of human grandmasters, and in the unsupervised learning phase these evolved organisms are further improved upon by means of coevolution. While past attempts succeeded in creating a grandmaster-level program by mimicking the behavior of existing computer chess programs, this paper presents the first successful attempt at evolving a state-of-the-art evaluation function by learning only from databases of games played by humans. Our results demonstrate that the evolved program outperforms a two-time World Computer Chess Champion.

Genetic Algorithms for Mentor-Assisted Evaluation Function Optimization

Abstract: In this paper we demonstrate how genetic algorithms can be used to reverse engineer an evaluation function's parameters for computer chess. Our results show that using an appropriate mentor, we can evolve a program that is on par with top tournament-playing chess programs, outperforming a two-time World Computer Chess Champion. This performance gain is achieved by evolving a program with a smaller number of parameters in its evaluation function to mimic the behavior of a superior mentor which uses a more extensive evaluation function. In principle, our mentor-assisted approach could be used in a wide range of problems for which appropriate mentors are available.