We present a new technique called “t-SNE” that visualizes high-dimensional data by giving each datapoint a location in a two or three-dimensional map. The technique is a variation of Stochastic Neighbor Embedding (Hinton and Roweis, 2002) that is much easier to optimize, and produces significantly better visualizations by reducing the tendency to crowd points together in the center of the map. t-SNE is better than existing techniques at creating a single map that reveals structure at many different scales. This is particularly important for high-dimensional data that lie on several different, but related, low-dimensional manifolds, such as images of objects from multiple classes seen from multiple viewpoints. For visualizing the structure of very large datasets, we show how t-SNE can use random walks on neighborhood graphs to allow the implicit structure of all of the data to influence the way in which a subset of the data is displayed. We illustrate the performance of t-SNE on a wide variety of datasets and compare it with many other non-parametric visualization techniques, including Sammon mapping, Isomap, and Locally Linear Embedding. The visualizations produced by t-SNE are significantly better than those produced by the other techniques on almost all of the datasets.

/pdf/visualizing-data-using-t-sne-2w1n304iq0.pdf

Visualizing Data using t-SNE

Three parallel algorithms for classical molecular dynamics are presented. The first assigns each processor a fixed subset of atoms; the second assigns each a fixed subset of inter-atomic forces to compute; the third assigns each a fixed spatial region. The algorithms are suitable for molecular dynamics models which can be difficult to parallelize efficiently—those with short-range forces where the neighbors of each atom change rapidly. They can be implemented on any distributed-memory parallel machine which allows for message-passing of data between independently executing processors. The algorithms are tested on a standard Lennard-Jones benchmark problem for system sizes ranging from 500 to 100,000,000 atoms on several parallel supercomputers--the nCUBE 2, Intel iPSC/860 and Paragon, and Cray T3D. Comparing the results to the fastest reported vectorized Cray Y-MP and C90 algorithm shows that the current generation of parallel machines is competitive with conventional vector supercomputers even for small problems. For large problems, the spatial algorithm achieves parallel efficiencies of 90% and a 1840-node Intel Paragon performs up to 165 faster than a single Cray C9O processor. Trade-offs between the three algorithms and guidelines for adapting them to more complex molecular dynamics simulations are also discussed.

Fast parallel algorithms for short-range molecular dynamics

https://axon.cs.byu.edu/~martinez/classes/678/Presentations/Yao.pdf

Extreme learning machine: Theory and applications

Probability Distributions.- Linear Models for Regression.- Linear Models for Classification.- Neural Networks.- Kernel Methods.- Sparse Kernel Machines.- Graphical Models.- Mixture Models and EM.- Approximate Inference.- Sampling Methods.- Continuous Latent Variables.- Sequential Data.- Combining Models.

Pattern Recognition and Machine Learning

Data Mining - Concepts and Techniques.

In this paper, a global methodology for variable selection is presented. This methodology is optimizing the Nonparametric Noise Estimation (NNE) provided by Delta Test. The 3 steps of the methodology are Variable Selection (VS), Scaling and Projection. The methodology is applies to two examples: the Boston Housing database and a financial data set. It is shown that the proposed methodology provides better input variables than an exhaustive search. Furthermore, interpretability of the results is improved.

Variable Selection for Financial Modeling

Editorial: Advances in extreme learning machines (ELM2012)

In this article, the moments of nearest neighbor distance distributions are examined. While the asymptotic form of such moments is well-known, the boundary effect has this far resisted a rigorous analysis. Our goal is to develop a new technique that allows a closed-form high order expansion, where the boundaries are taken into account up to the first order. The resulting theoretical predictions are tested via simulations and found to be much more accurate than the first order approximation obtained by neglecting the boundaries.

While our results are of theoretical interest, they definitely also have important applications in statistics and physics. As a concrete example, we mention estimating Renyi entropies of probability distributions. Moreover, the algebraic technique developed may turn out to be useful in other, related problems including estimation of the Shannon differential entropy.© 2010 Wiley Periodicals, Inc. Random Struct. Alg., 2010

/pdf/a-boundary-corrected-expansion-of-the-moments-of-nearest-5civms0fib.pdf

A boundary corrected expansion of the moments of nearest neighbor distributions

This paper presents a novel library for Extreme Learning Machines (ELM) called Scikit-ELM (https://github.com/akusok/scikit-elm, https://scikit-elm.readthedocs.io). Usability and flexibility of the approach are the main focus points in this work, achieved primarily through a tight integration with Scikit-Learn, a de facto industry standard library in Machine Learning outside Deep Learning. Methodological advances enable great flexibility in dynamic addition of new classes to a trained model, or by allowing a model to forget previously learned data.

Scikit-ELM: An Extreme Learning Machine Toolbox for Dynamic and Scalable Learning

In this paper a study is described for several approaches to the identification of models for the temperature within the melter portion of a glass furnace. The focus is on developing models from the gas input to the throat (melter outlet) temperature. Conventional linear techniques for system identification proved to be inadequate for this problem, but proved useful as baseline comparisons for further studies involving nonlinear techniques from intelligent control for model building. Various combinations of input and output variables in a variety of model structures using fuzzy and neuro-fuzzy system modeling approaches are developed, and comparisons are drawn. Approaches reported on here investigate nonlinear Takagi-Sugeno (TS) fuzzy model formulations, where a linear-in-the-parameter identification problem is formulated for various combinations of measured variables and system delays. A fuzzy-neuro formulation is then discussed for parameter selection in the TS model structure while simultaneously optimizing the membership functions associated with the inputs of the TS fuzzy system. Simulation results for data collected from an operating glass furnace process are presented.

Amaury Lendasse

Papers

Variable Selection for Financial Modeling

Editorial: Advances in extreme learning machines (ELM2012)

A boundary corrected expansion of the moments of nearest neighbor distributions

Scikit-ELM: An Extreme Learning Machine Toolbox for Dynamic and Scalable Learning

Identification of fuzzy models for a glass furnace process