Journal ArticleDOI
Apache Spark: a unified engine for big data processing
Matei Zaharia,Reynold Xin,Patrick Wendell,Tathagata Das,Michael Armbrust,Ankur Dave,Xiangrui Meng,Josh Rosen,Shivaram Venkataraman,Michael J. Franklin,Ali Ghodsi,Joseph E. Gonzalez,Scott Shenker,Ion Stoica +13 more
Reads0
Chats0
TLDR
This open source computing framework unifies streaming, batch, and interactive big data workloads to unlock new applications.Abstract:
This open source computing framework unifies streaming, batch, and interactive big data workloads to unlock new applicationsread more
Citations
More filters
Journal ArticleDOI
Understanding gender differences in professional European football through machine learning interpretability and match actions data.
TL;DR: In this article, the main differential features of European male and female football players in match actions data under the assumption of finding significant differences and established patterns between genders were evaluated and a methodology for unbiased feature extraction and objective analysis is presented based on data integration and machine learning explainability algorithms.
Journal ArticleDOI
Scedar: A scalable Python package for single-cell RNA-seq exploratory data analysis.
TL;DR: The scedar package is a scalable Python package for scRNA-seq exploratory data analysis that provides a convenient and reliable interface for performing visualization, imputation of gene dropouts, detection of rare transcriptomic profiles, and clustering on large-scale sc RNA-seq datasets.
Journal ArticleDOI
Comparative Study of Binary Classification Methods to Analyze a Massive Dataset on Virtual Machine
Neelam Naik,Seema Purohit +1 more
TL;DR: The comparative study of binary classification methods such as decision tree, gradient boosted tree and random forest tree is performed to judge their performances on the basis of defined parameters and it is found that Random forest tree performs best among all three algorithms for the considered dataset.
Journal ArticleDOI
Mitigating the Impact of Data Sampling on Social Media Analysis and Mining
TL;DR: This article explores a combination of spectral clustering, locality-sensitive hashing, latent Dirichlet allocation (LDA) topic modeling, and differential equation modeling to mitigate the impact of sampling on social media data analysis, in particular on detecting real-world events and predicting information diffusion.
Proceedings ArticleDOI
A NoSQL Database Approach for Modeling Heterogeneous and Semi-Structured Information
TL;DR: A robust analytics framework is built by integrating Apache Spark with Apache Cassandra and in following utilize data mining techniques for presenting a model capable of predicting the relationship between tourist arrivals and nights spent in Greece.
References
More filters
Journal ArticleDOI
MapReduce: simplified data processing on large clusters
Jeffrey Dean,Sanjay Ghemawat +1 more
TL;DR: This paper presents the implementation of MapReduce, a programming model and an associated implementation for processing and generating large data sets that runs on a large cluster of commodity machines and is highly scalable.
Proceedings Article
Resilient distributed datasets: a fault-tolerant abstraction for in-memory cluster computing
Matei Zaharia,Mosharaf Chowdhury,Tathagata Das,Ankur Dave,Justin Ma,Murphy McCauley,Michael J. Franklin,Scott Shenker,Ion Stoica +8 more
TL;DR: Resilient Distributed Datasets is presented, a distributed memory abstraction that lets programmers perform in-memory computations on large clusters in a fault-tolerant manner and is implemented in a system called Spark, which is evaluated through a variety of user applications and benchmarks.
Journal ArticleDOI
A bridging model for parallel computation
TL;DR: The bulk-synchronous parallel (BSP) model is introduced as a candidate for this role, and results quantifying its efficiency both in implementing high-level language features and algorithms, as well as in being implemented in hardware.
Proceedings ArticleDOI
Pregel: a system for large-scale graph processing
Grzegorz Malewicz,Matthew H. Austern,Aart J. C. Bik,James C. Dehnert,Ilan Horn,Naty Leiser,Grzegorz Czajkowski +6 more
TL;DR: A model for processing large graphs that has been designed for efficient, scalable and fault-tolerant implementation on clusters of thousands of commodity computers, and its implied synchronicity makes reasoning about programs easier.
Proceedings ArticleDOI
Dryad: distributed data-parallel programs from sequential building blocks
TL;DR: The Dryad execution engine handles all the difficult problems of creating a large distributed, concurrent application: scheduling the use of computers and their CPUs, recovering from communication or computer failures, and transporting data between vertices.