Open AccessProceedings Article
Map-Reduce for Machine Learning on Multicore
Cheng-Tao Chu,Sang K. Kim,Yi-an Lin,Yuanyuan Yu,Gary Bradski,Kunle Olukotun,Andrew Y. Ng +6 more
- Vol. 19, pp 281-288
TLDR
This work shows that algorithms that fit the Statistical Query model can be written in a certain "summation form," which allows them to be easily parallelized on multicore computers and shows basically linear speedup with an increasing number of processors.Abstract:
We are at the beginning of the multicore era. Computers will have increasingly many cores (processors), but there is still no good programming framework for these architectures, and thus no simple and unified way for machine learning to take advantage of the potential speed up. In this paper, we develop a broadly applicable parallel programming method, one that is easily applied to many different learning algorithms. Our work is in distinct contrast to the tradition in machine learning of designing (often ingenious) ways to speed up a single algorithm at a time. Specifically, we show that algorithms that fit the Statistical Query model [15] can be written in a certain "summation form," which allows them to be easily parallelized on multicore computers. We adapt Google's map-reduce [7] paradigm to demonstrate this parallel speed up technique on a variety of learning algorithms including locally weighted linear regression (LWLR), k-means, logistic regression (LR), naive Bayes (NB), SVM, ICA, PCA, gaussian discriminant analysis (GDA), EM, and backpropagation (NN). Our experimental results show basically linear speedup with an increasing number of processors.read more
Citations
More filters
Proceedings ArticleDOI
Beyond Simple Integration of RDBMS and MapReduce -- Paving the Way toward a Unified System for Big Data Analytics: Vision and Progress
Xiongpai Qin,Huiju Wang,Furong Li,Baoyao Zhou,Yu Cao,Cuiping Li,Hong Chen,Xuan Zhou,Xiaoyong Du,Shan Wang +9 more
TL;DR: The authors envision that the two techniques are fusing into a unified system for big data analytics, which RDBMS enjoys high performance of relational data processing, which MapReduce needs to catch up.
Journal ArticleDOI
Merging weighted SVMs for parallel incremental learning.
TL;DR: Both theoretical and experimental studies show the equivalence of the proposed algorithm to batch wESVM in terms of learning effectiveness, and the algorithm demonstrates desired scalability and clear speed advantages to batch retraining.
Hadoop's Overload Tolerant Design Exacerbates Failure Detection and Recovery
Florin Dinu,T. S. Eugene Ng +1 more
TL;DR: It is shown that Hadoop, which couples failure detection and recovery with overload handling into a conservative design with conservative parameter choices, is oftentimes slow in reacting to failures and also exhibits large variations in response time under failures.
Journal ArticleDOI
Intelligent Reconfigurable Method of Cloud Computing Resources for Multimedia Data Delivery
TL;DR: This study proposes a method of designing a scheme for applying MapReduce of the FP-Growth algorithm which is one of data mining methods based on the Hadoop platform at the stage of IaaS (Infrastructure As a Service) including CPU, networking and storages.
Proceedings ArticleDOI
Framework for multi threads execution of data mining algorithms
TL;DR: The present paper describes the framework for creating data mining algorithms from thread-safe functional blocks, which allows create new datamining algorithms from existing blocks and improves the existing algorithms by optimizing single blocks or the whole structure of the algorithms.
References
More filters
Journal ArticleDOI
Learning representations by back-propagating errors
TL;DR: Back-propagation repeatedly adjusts the weights of the connections in the network so as to minimize a measure of the difference between the actual output vector of the net and the desired output vector, which helps to represent important features of the task domain.
Journal ArticleDOI
MapReduce: simplified data processing on large clusters
Jeffrey Dean,Sanjay Ghemawat +1 more
TL;DR: This paper presents the implementation of MapReduce, a programming model and an associated implementation for processing and generating large data sets that runs on a large cluster of commodity machines and is highly scalable.
Journal ArticleDOI
An information-maximization approach to blind separation and blind deconvolution
TL;DR: It is suggested that information maximization provides a unifying framework for problems in "blind" signal processing and dependencies of information transfer on time delays are derived.
Journal ArticleDOI
Principal component analysis
TL;DR: Principal Component Analysis is a multivariate exploratory analysis method useful to separate systematic variation from noise and to define a space of reduced dimensions that preserve noise.