Open AccessProceedings Article
Map-Reduce for Machine Learning on Multicore
Cheng-Tao Chu,Sang K. Kim,Yi-an Lin,Yuanyuan Yu,Gary Bradski,Kunle Olukotun,Andrew Y. Ng +6 more
- Vol. 19, pp 281-288
TLDR
This work shows that algorithms that fit the Statistical Query model can be written in a certain "summation form," which allows them to be easily parallelized on multicore computers and shows basically linear speedup with an increasing number of processors.Abstract:
We are at the beginning of the multicore era. Computers will have increasingly many cores (processors), but there is still no good programming framework for these architectures, and thus no simple and unified way for machine learning to take advantage of the potential speed up. In this paper, we develop a broadly applicable parallel programming method, one that is easily applied to many different learning algorithms. Our work is in distinct contrast to the tradition in machine learning of designing (often ingenious) ways to speed up a single algorithm at a time. Specifically, we show that algorithms that fit the Statistical Query model [15] can be written in a certain "summation form," which allows them to be easily parallelized on multicore computers. We adapt Google's map-reduce [7] paradigm to demonstrate this parallel speed up technique on a variety of learning algorithms including locally weighted linear regression (LWLR), k-means, logistic regression (LR), naive Bayes (NB), SVM, ICA, PCA, gaussian discriminant analysis (GDA), EM, and backpropagation (NN). Our experimental results show basically linear speedup with an increasing number of processors.read more
Citations
More filters
Journal ArticleDOI
Using statistical text classification to identify health information technology incidents.
TL;DR: Statistical text classification appears to be a feasible method for identifying HIT reports within large databases of incidents and semi-supervised learning may be necessary when applying machine learning to big data analysis of patient safety incidents.
Journal ArticleDOI
Data and task parallelism in ILP using MapReduce
TL;DR: This paper examines the applicability to ILP of a popular distributed computing approach that provides a uniform way for performing data and task parallel computations in ILP, and shows how the MapReduce approach can be used to perform the coverage-test, and to perform multiple searches required by a greedy set-covering algorithm used by some popular ILP systems.
Proceedings ArticleDOI
Accelerating Spatial Data Processing with MapReduce
TL;DR: Methods as follows are presented, a splitting method for balancing workload, a pending file structure and redundant data partition dealing with relation between spatial objects, and a strip-based two-direction plane sweeping algorithm for computation accelerating that outperform the traditional one on DBMS.
Posted Content
Scaling Datalog for Machine Learning on Big Data
Yingyi Bu,Vinayak Borkar,Michael J. Carey,Joshua Rosen,Neoklis Polyzotis,Tyson Condie,Markus Weimer,Raghu Ramakrishnan +7 more
TL;DR: This paper argues for the use of recursive queries to program a variety of machine learning systems using database query optimization techniques to identify effective execution plans, and the resulting runtime plans can be executed on a single unified data-parallel query processing engine.
Proceedings ArticleDOI
Coded Distributed Computing: Straggling Servers and Multistage Dataflows
TL;DR: A unified coding scheme is described that superimposes CDC with the Maximum-Distance-Separable coding on computation tasks, which allows a flexible tradeoff between computation latency and communication load.
References
More filters
Journal ArticleDOI
Learning representations by back-propagating errors
TL;DR: Back-propagation repeatedly adjusts the weights of the connections in the network so as to minimize a measure of the difference between the actual output vector of the net and the desired output vector, which helps to represent important features of the task domain.
Journal ArticleDOI
MapReduce: simplified data processing on large clusters
Jeffrey Dean,Sanjay Ghemawat +1 more
TL;DR: This paper presents the implementation of MapReduce, a programming model and an associated implementation for processing and generating large data sets that runs on a large cluster of commodity machines and is highly scalable.
Journal ArticleDOI
An information-maximization approach to blind separation and blind deconvolution
TL;DR: It is suggested that information maximization provides a unifying framework for problems in "blind" signal processing and dependencies of information transfer on time delays are derived.
Journal ArticleDOI
Principal component analysis
TL;DR: Principal Component Analysis is a multivariate exploratory analysis method useful to separate systematic variation from noise and to define a space of reduced dimensions that preserve noise.