Open AccessProceedings Article
Map-Reduce for Machine Learning on Multicore
Cheng-Tao Chu,Sang K. Kim,Yi-an Lin,Yuanyuan Yu,Gary Bradski,Kunle Olukotun,Andrew Y. Ng +6 more
- Vol. 19, pp 281-288
TLDR
This work shows that algorithms that fit the Statistical Query model can be written in a certain "summation form," which allows them to be easily parallelized on multicore computers and shows basically linear speedup with an increasing number of processors.Abstract:
We are at the beginning of the multicore era. Computers will have increasingly many cores (processors), but there is still no good programming framework for these architectures, and thus no simple and unified way for machine learning to take advantage of the potential speed up. In this paper, we develop a broadly applicable parallel programming method, one that is easily applied to many different learning algorithms. Our work is in distinct contrast to the tradition in machine learning of designing (often ingenious) ways to speed up a single algorithm at a time. Specifically, we show that algorithms that fit the Statistical Query model [15] can be written in a certain "summation form," which allows them to be easily parallelized on multicore computers. We adapt Google's map-reduce [7] paradigm to demonstrate this parallel speed up technique on a variety of learning algorithms including locally weighted linear regression (LWLR), k-means, logistic regression (LR), naive Bayes (NB), SVM, ICA, PCA, gaussian discriminant analysis (GDA), EM, and backpropagation (NN). Our experimental results show basically linear speedup with an increasing number of processors.read more
Citations
More filters
Book ChapterDOI
An empirical study of massively parallel bayesian networks learning for sentiment extraction from unstructured text
TL;DR: A parallel algorithm for BN (Bayesian Networks) structure leaning from large-scale dateset by using a MapReduce cluster is presented and is able to select a parsimonious feature set with substantially fewer predictor variables than in the full data set and leads to better predictions about sentiment orientations than several usually used methods.
Journal ArticleDOI
Map / Reduce Deisgn and Implementation of Apriori Alogirthm for handling voluminous data-sets
TL;DR: This paper focuses on map/reduce design and implementation of Apriori algorithm for structured data analysis, which stands as an elementary foundation to supervised learning, which encompasses classifier and feature extraction methods.
Book ChapterDOI
Diversity-Driven Widening
TL;DR: This paper presents a more in-depth analysis of the concept of Widened Data Mining, which aims at reducing the impact of greedy heuristics by exploring more than just one suitable solution at each step by focusing on how diversity considerations can substantially improve results.
Journal ArticleDOI
Machine Learning Patterns for Neuroimaging-Genetic Studies in the Cloud
Benoit Da Mota,Radu Tudoran,Alexandru Costan,Gaël Varoquaux,Goetz Brasche,Patricia J. Conrod,Herve Lemaitre,Tomáš Paus,Tomáš Paus,Tomáš Paus,Marcella Rietschel,Vincent Frouin,Jean-Baptiste Poline,Gabriel Antoniu,Bertrand Thirion +14 more
TL;DR: A scalable analysis tool that can deal with non-parametric statistics on high-dimensional data by combining a MapReduce framework (TomusBLOB) with machine learning algorithms (Scikit-learn library) and demonstrating the scalability and the reliability of the framework in the cloud with a 2 weeks deployment on hundreds of virtual machines.
Posted Content
Distributed linear regression by averaging
Edgar Dobriban,Yue Sheng +1 more
TL;DR: The performance loss in one-step and iterative weighted parameter averaging in statistical linear models under data parallelism is found, and a new calculus of deterministic equivalents as a tool of broader interest is developed.
References
More filters
Journal ArticleDOI
Learning representations by back-propagating errors
TL;DR: Back-propagation repeatedly adjusts the weights of the connections in the network so as to minimize a measure of the difference between the actual output vector of the net and the desired output vector, which helps to represent important features of the task domain.
Journal ArticleDOI
MapReduce: simplified data processing on large clusters
Jeffrey Dean,Sanjay Ghemawat +1 more
TL;DR: This paper presents the implementation of MapReduce, a programming model and an associated implementation for processing and generating large data sets that runs on a large cluster of commodity machines and is highly scalable.
Journal ArticleDOI
An information-maximization approach to blind separation and blind deconvolution
TL;DR: It is suggested that information maximization provides a unifying framework for problems in "blind" signal processing and dependencies of information transfer on time delays are derived.
Journal ArticleDOI
Principal component analysis
TL;DR: Principal Component Analysis is a multivariate exploratory analysis method useful to separate systematic variation from noise and to define a space of reduced dimensions that preserve noise.