We present a new technique called “t-SNE” that visualizes high-dimensional data by giving each datapoint a location in a two or three-dimensional map. The technique is a variation of Stochastic Neighbor Embedding (Hinton and Roweis, 2002) that is much easier to optimize, and produces significantly better visualizations by reducing the tendency to crowd points together in the center of the map. t-SNE is better than existing techniques at creating a single map that reveals structure at many different scales. This is particularly important for high-dimensional data that lie on several different, but related, low-dimensional manifolds, such as images of objects from multiple classes seen from multiple viewpoints. For visualizing the structure of very large datasets, we show how t-SNE can use random walks on neighborhood graphs to allow the implicit structure of all of the data to influence the way in which a subset of the data is displayed. We illustrate the performance of t-SNE on a wide variety of datasets and compare it with many other non-parametric visualization techniques, including Sammon mapping, Isomap, and Locally Linear Embedding. The visualizations produced by t-SNE are significantly better than those produced by the other techniques on almost all of the datasets.

/pdf/visualizing-data-using-t-sne-2w1n304iq0.pdf

Visualizing Data using t-SNE

Three parallel algorithms for classical molecular dynamics are presented. The first assigns each processor a fixed subset of atoms; the second assigns each a fixed subset of inter-atomic forces to compute; the third assigns each a fixed spatial region. The algorithms are suitable for molecular dynamics models which can be difficult to parallelize efficiently—those with short-range forces where the neighbors of each atom change rapidly. They can be implemented on any distributed-memory parallel machine which allows for message-passing of data between independently executing processors. The algorithms are tested on a standard Lennard-Jones benchmark problem for system sizes ranging from 500 to 100,000,000 atoms on several parallel supercomputers--the nCUBE 2, Intel iPSC/860 and Paragon, and Cray T3D. Comparing the results to the fastest reported vectorized Cray Y-MP and C90 algorithm shows that the current generation of parallel machines is competitive with conventional vector supercomputers even for small problems. For large problems, the spatial algorithm achieves parallel efficiencies of 90% and a 1840-node Intel Paragon performs up to 165 faster than a single Cray C9O processor. Trade-offs between the three algorithms and guidelines for adapting them to more complex molecular dynamics simulations are also discussed.

Fast parallel algorithms for short-range molecular dynamics

https://axon.cs.byu.edu/~martinez/classes/678/Presentations/Yao.pdf

Extreme learning machine: Theory and applications

Probability Distributions.- Linear Models for Regression.- Linear Models for Classification.- Neural Networks.- Kernel Methods.- Sparse Kernel Machines.- Graphical Models.- Mixture Models and EM.- Approximate Inference.- Sampling Methods.- Continuous Latent Variables.- Sequential Data.- Combining Models.

Pattern Recognition and Machine Learning

Data Mining - Concepts and Techniques.

The bootstrap resampling method may be efficiently used to estimate the generalization error of a family of nonlinear regression models, as artificial neural networks. The main difficulty associated with the bootstrap in real-world applications is the high computation load. In this paper we propose a simple procedure based on empirical evidence, to considerably reduce the computation time needed to estimate the generalization error of a family of models of increasing number of parameters.

/pdf/fast-approximation-of-the-bootstrap-for-model-selection-2l6z0sy0hd.pdf

Fast approximation of the bootstrap for model selection

This paper presents a novel dimensionality reduction technique: ELM-SOM. This technique preserves the intrinsic quality of Self-Organizing Maps (SOM): it is nonlinear and suitable for big data. It also brings continuity to the projection using two Extreme Learning Machine (ELM) models, the first one to perform the dimensionality reduction and the second one to perform the reconstruction. ELM-SOM is tested successfully on six diverse datasets. Regarding reconstruction error, ELM-SOM is comparable to SOM while bringing continuity.

ELM-SOM: A Continuous Self-Organizing Map for Visualization

1- University of Ja´en - Department of InformaticsSpain2- University of Granada - Department of Computer Technology and ArchitectureSpain3- Helsinki University of Technology - Information and Computer Science DepartmentFinlandAbstract The problem of selecting the patterns to be learned by anymodel is usually not considered by the time of designing the concretemodel but as a preprocessing step Information theory provides a robusttheoretical framework for performing input variable selection thanks to theconcept of mutual information Recently the computation of the mutualinformation for regression tasks has been proposed so this paper presentsa new application of the concept of mutual information not to select thevariables but to decide which prototypes should belong to the training dataset in regression problems The proposed methodology consists in decidingif a prototype should belong or not to the training set using as criteria theestimation of the mutual information between the variables The noveltyof the approach is to focus in prototype selection for regression problemsinstead of classiﬁcation as the majority of the literature deals only withthe last one Other element that distinguishes this work from others isthat it is not proposed as an outlier identiﬁcator but as an algorithm thatdetermines the best subset of input vectors by the time of building a modelto approximate it As the experiment section shows, this new method isable to identify a high percentage of the real data set when it is appliedto a highly distorted data sets

https://www.elen.ucl.ac.be/Proceedings/esann/esannpdf/es2009-126.pdf

Applying Mutual Information for Prototype or Instance Selection in Regression Problems

Image-based Classification of Websites

A novel particle swarm optimization based selective ensemble (PSOSEN) of online sequential extreme learning machine (OS-ELM) is proposed. It is based on the original OS-ELM with an adaptive selective ensemble framework. Two novel insights are proposed in this paper. First, a novel selective ensemble algorithm referred to as particle swarm optimization selective ensemble is proposed, noting that PSOSEN is a general selective ensemble method which is applicable to any learning algorithms, including batch learning and online learning. Second, an adaptive selective ensemble framework for online learning is designed to balance the accuracy and speed of the algorithm. Experiments for both regression and classification problems with UCI data sets are carried out. Comparisons between OS-ELM, simple ensemble OS-ELM (EOS-ELM), genetic algorithm based selective ensemble (GASEN) of OS-ELM, and the proposed particle swarm optimization based selective ensemble of OS-ELM empirically show that the proposed algorithm achieves good generalization performance and fast learning speed.

/pdf/particle-swarm-optimization-based-selective-ensemble-of-3u23r2boqt.pdf

Amaury Lendasse

Papers

Fast approximation of the bootstrap for model selection

ELM-SOM: A Continuous Self-Organizing Map for Visualization

Applying Mutual Information for Prototype or Instance Selection in Regression Problems

Image-based Classification of Websites

Particle Swarm Optimization Based Selective Ensemble of Online Sequential Extreme Learning Machine