We present a new technique called “t-SNE” that visualizes high-dimensional data by giving each datapoint a location in a two or three-dimensional map. The technique is a variation of Stochastic Neighbor Embedding (Hinton and Roweis, 2002) that is much easier to optimize, and produces significantly better visualizations by reducing the tendency to crowd points together in the center of the map. t-SNE is better than existing techniques at creating a single map that reveals structure at many different scales. This is particularly important for high-dimensional data that lie on several different, but related, low-dimensional manifolds, such as images of objects from multiple classes seen from multiple viewpoints. For visualizing the structure of very large datasets, we show how t-SNE can use random walks on neighborhood graphs to allow the implicit structure of all of the data to influence the way in which a subset of the data is displayed. We illustrate the performance of t-SNE on a wide variety of datasets and compare it with many other non-parametric visualization techniques, including Sammon mapping, Isomap, and Locally Linear Embedding. The visualizations produced by t-SNE are significantly better than those produced by the other techniques on almost all of the datasets.

/pdf/visualizing-data-using-t-sne-2w1n304iq0.pdf

Visualizing Data using t-SNE

Three parallel algorithms for classical molecular dynamics are presented. The first assigns each processor a fixed subset of atoms; the second assigns each a fixed subset of inter-atomic forces to compute; the third assigns each a fixed spatial region. The algorithms are suitable for molecular dynamics models which can be difficult to parallelize efficiently—those with short-range forces where the neighbors of each atom change rapidly. They can be implemented on any distributed-memory parallel machine which allows for message-passing of data between independently executing processors. The algorithms are tested on a standard Lennard-Jones benchmark problem for system sizes ranging from 500 to 100,000,000 atoms on several parallel supercomputers--the nCUBE 2, Intel iPSC/860 and Paragon, and Cray T3D. Comparing the results to the fastest reported vectorized Cray Y-MP and C90 algorithm shows that the current generation of parallel machines is competitive with conventional vector supercomputers even for small problems. For large problems, the spatial algorithm achieves parallel efficiencies of 90% and a 1840-node Intel Paragon performs up to 165 faster than a single Cray C9O processor. Trade-offs between the three algorithms and guidelines for adapting them to more complex molecular dynamics simulations are also discussed.

Fast parallel algorithms for short-range molecular dynamics

https://axon.cs.byu.edu/~martinez/classes/678/Presentations/Yao.pdf

Extreme learning machine: Theory and applications

Probability Distributions.- Linear Models for Regression.- Linear Models for Classification.- Neural Networks.- Kernel Methods.- Sparse Kernel Machines.- Graphical Models.- Mixture Models and EM.- Approximate Inference.- Sampling Methods.- Continuous Latent Variables.- Sequential Data.- Combining Models.

Pattern Recognition and Machine Learning

Data Mining - Concepts and Techniques.

In this paper is described the original (basic) Extreme Learning Machine (ELM). Properties like robustness and sensitivity to variable selection are studied. Several extensions of the original ELM are then presented and compared. Firstly, Tikhonov-Regularized Optimally-Pruned Extreme Learning Machine (TROP-ELM) is summarized as an improvement of the Optimally-Pruned Extreme Learning Machine (OP-ELM) in the form of a L2 regularization penalty applied within the OP-ELM. Secondly, a Methodology to Linearly Ensemble ELM (ELM-ELM) is presented in order to improve the performance of the original ELM. These methodologies (TROP-ELM and ELM-ELM) are tested against state of the art methods such as Support Vector Machines or Gaussian Processes and the original ELM and OP-ELM, on ten different data sets. A specific experiment to test the sensitivity of these methodologies to variable selection is also presented.

Extreme learning machine: a robust modeling technique? yes!

Nowadays, image recognition has become a highly active research topic in cognitive computation community, due to its many potential applications. Generally, the image recognition task involves two subtasks: image representation and image classification. Most feature extraction approaches for image representation developed so far regard independent component analysis (ICA) as one of the essential means. However, ICA has been hampered by its extremely expensive computational cost in real-time implementation. To address this problem, a fast cognitive computational scheme for image recognition is presented in this paper, which combines ICA and the extreme learning machine (ELM) algorithm. It tries to solve the image recognition problem at a much faster speed by using ELM not only in image classification but also in feature extraction for image representation. As an example, our proposed approach is applied to the face image recognition with detailed analysis. Firstly, common feature hypothesis is introduced to extract the common visual features from universal images by the traditional ICA model in the offline recognition process, and then ELM is used to simulate ICA for the purpose of facial feature extraction in the online recognition process. Lastly, the resulting independent feature representation of the face images extracted by ELM rather than ICA will be fed into the ELM classifier, which is composed of numerous single hidden layer feed-forward networks. Experimental results on Yale face database and MNIST digit database have shown the good performance of our proposed approach, which could be comparable to the state-of-the-art techniques at a much faster speed.

Fast Image Recognition Based on Independent Component Analysis and Extreme Learning Machine

SOM-ELM-Self-Organized Clustering using ELM

This paper presents a short introduction to the Reservoir Computing and Extreme Learning Machine main ideas and developments. While both methods make use of Neural Networks and Random Projections, Reservoir Computing allows the network to have a recurrent structure, while the Extreme Learning Machine is a Feedforward neural network only. Some state of the art techniques are briefly presented and this special session papers are finally briefly described, in the terms of this introductory paper.

Amaury Lendasse

Papers

Extreme learning machine: a robust modeling technique? yes!

Fast Image Recognition Based on Independent Component Analysis and Extreme Learning Machine

SOM-ELM-Self-Organized Clustering using ELM

Machine Learning Techniques based on Random Projections.

Larsen-elm