Open AccessJournal Article
MLlib: machine learning in apache spark
Xiangrui Meng,Joseph K. Bradley,Burak Yavuz,Evan R. Sparks,Shivaram Venkataraman,Davies Liu,Jeremy Freeman,DB Tsai,Manish Amde,Sean Owen,Doris Xin,Reynold Xin,Michael J. Franklin,Reza Bosagh Zadeh,Matei Zaharia,Ameet Talwalkar +15 more
Reads0
Chats0
TLDR
MLlib as mentioned in this paper is an open-source distributed machine learning library for Apache Spark that provides efficient functionality for a wide range of learning settings and includes several underlying statistical, optimization, and linear algebra primitives.Abstract:
Apache Spark is a popular open-source platform for large-scale data processing that is well-suited for iterative machine learning tasks. In this paper we present MLlib, Spark's open-source distributed machine learning library. MLLIB provides efficient functionality for a wide range of learning settings and includes several underlying statistical, optimization, and linear algebra primitives. Shipped with Spark, MLLIB supports several languages and provides a high-level API that leverages Spark's rich ecosystem to simplify the development of end-to-end machine learning pipelines. MLLIB has experienced a rapid growth due to its vibrant open-source community of over 140 contributors, and includes extensive documentation to support further growth and to let users quickly get up to speed.read more
Citations
More filters
Book ChapterDOI
Delay Prediction System for Large-Scale Railway Networks Based on Big Data Analytics
Luca Oneto,Emanuele Fumeo,Giorgio Clerico,Renzo Canepa,Federico Papa,Carlo Dambra,Nadia Mazzino,Davide Anguita +7 more
TL;DR: This paper proposes a fast learning algorithm for predicting train delays based on the Extreme Learning Machine that fully exploits the recent in-memory large-scale data processing technologies.
Journal Article
Detection and Extraction of Brain Hemorrhage on the CT-Scan Image using Hybrid Thresholding Method
TL;DR: This research will determine the area of the brain bleeding on each image slice CT - scan every patient, to detect and extract brain bleeding, so it can calculate the volume of thebrain bleeding.
Proceedings ArticleDOI
A Machine Learning Approach to Demographic Prediction using Geohashes
Avipsa Roy,Edzer Pebesma +1 more
TL;DR: This paper uses a machine learning approach to predict the gender and age of mobile phone users from a set of 3,252,950 anonymised GPS trajectories with 60,865 unique devices using a predictive model which is based upon the concept of Geohashes.
Proceedings ArticleDOI
AFrame: Extending DataFrames for Large-Scale Modern Data Analysis
TL;DR: The architecture of AFrame is presented, the underlying capabilities of AsterixDB that efficiently support modern data analytic operations are described, and an extensible micro-benchmark is introduced for use in evaluating DataFrame performance in both single-node and distributed settings via a collection of representative analytic operations.
Journal ArticleDOI
A Model and Survey of Distributed Data-Intensive Systems
TL;DR: This research presents a meta-analyses of the immune system’s response to machine learning and its applications in the context of social reinforcement learning and reinforcement learning.
References
More filters
Journal Article
Scikit-learn: Machine Learning in Python
Fabian Pedregosa,Gaël Varoquaux,Alexandre Gramfort,Vincent Michel,Bertrand Thirion,Olivier Grisel,Mathieu Blondel,Peter Prettenhofer,Ron Weiss,Vincent Dubourg,Jake Vanderplas,Alexandre Passos,David Cournapeau,Matthieu Brucher,Matthieu Perrot,Edouard Duchesnay +15 more
TL;DR: Scikit-learn is a Python module integrating a wide range of state-of-the-art machine learning algorithms for medium-scale supervised and unsupervised problems, focusing on bringing machine learning to non-specialists using a general-purpose high-level language.
Journal ArticleDOI
Latent dirichlet allocation
TL;DR: This work proposes a generative model for text and other collections of discrete data that generalizes or improves on several previous models including naive Bayes/unigram, mixture of unigrams, and Hofmann's aspect model.
Posted Content
Scikit-learn: Machine Learning in Python
Fabian Pedregosa,Gaël Varoquaux,Alexandre Gramfort,Vincent Michel,Bertrand Thirion,Olivier Grisel,Mathieu Blondel,Andreas Müller,Joel Nothman,Gilles Louppe,Peter Prettenhofer,Ron Weiss,Vincent Dubourg,Jake Vanderplas,Alexandre Passos,David Cournapeau,Matthieu Brucher,Matthieu Perrot,Edouard Duchesnay +18 more
TL;DR: Scikit-learn as mentioned in this paper is a Python module integrating a wide range of state-of-the-art machine learning algorithms for medium-scale supervised and unsupervised problems.
Proceedings Article
Latent Dirichlet Allocation
TL;DR: This paper proposed a generative model for text and other collections of discrete data that generalizes or improves on several previous models including naive Bayes/unigram, mixture of unigrams, and Hof-mann's aspect model, also known as probabilistic latent semantic indexing (pLSI).
Journal ArticleDOI
MapReduce: simplified data processing on large clusters
Jeffrey Dean,Sanjay Ghemawat +1 more
TL;DR: This paper presents the implementation of MapReduce, a programming model and an associated implementation for processing and generating large data sets that runs on a large cluster of commodity machines and is highly scalable.