Journal ArticleDOI
Apache Spark: a unified engine for big data processing
Matei Zaharia,Reynold Xin,Patrick Wendell,Tathagata Das,Michael Armbrust,Ankur Dave,Xiangrui Meng,Josh Rosen,Shivaram Venkataraman,Michael J. Franklin,Ali Ghodsi,Joseph E. Gonzalez,Scott Shenker,Ion Stoica +13 more
TLDR
This open source computing framework unifies streaming, batch, and interactive big data workloads to unlock new applications.Abstract:
This open source computing framework unifies streaming, batch, and interactive big data workloads to unlock new applicationsread more
Citations
More filters
Book ChapterDOI
Actionable Pattern Discovery for Tweet Emotions
TL;DR: This work focuses on extracting Action Rules with respect to the Emotion class from user tweets, which discovers actionable recommendations, which suggests ways to alter the user’s emotion to a better or more positive state.
Proceedings ArticleDOI
High Performance Data Engineering Everywhere
Chathura Widanage,Niranda Perera,Vibhatha Abeykoon,Supun Kamburugamuve,Thejaka Amila Kanewala,Hasara Maithree,Pulasthi Wickramasinghe,Ahmet Uyar,Gurhan Gunduz,Geoffrey C. Fox +9 more
TL;DR: Cylon as discussed by the authors is an open-source high performance distributed data processing library that can be seamlessly integrated with existing Big Data and AI/ML frameworks, which can be used as a library to existing applications or operate as a standalone framework.
Proceedings ArticleDOI
Apache Spark and Apache Ignite Performance Analysis
Cristiana-Stefania Stan,Adrian-Eduard Pandelica,Vlad-Andrei Zamfir,Roxana-Gabriela Stan,Catalin Negru +4 more
TL;DR: This paper compares two frameworks Apache Spark and Ignite that are used for data processing and shows that Spark achieved better performance than Ignite.
Proceedings ArticleDOI
Security-Performance Trade-offs of Kubernetes Container Runtimes
TL;DR: In this article, the authors compare the performance of three runtimes in the same Kubernetes cluster, the security focused Kata and gVisor, as well as the default runC, and demonstrate that runC outperforms the more secure alternatives up to 5x, while Kata executes container up to 1.6x faster than GVisor.
Journal ArticleDOI
Non-Query-Based Pattern Mining and Sentiment Analysis for Massive Microblogging Online Texts
TL;DR: A new approach of a non-query based system that combines association rules, generalized rules and sentiment analysis in order to catalogue and discover opinion patterns in the social network Twitter.
References
More filters
Journal ArticleDOI
MapReduce: simplified data processing on large clusters
Jeffrey Dean,Sanjay Ghemawat +1 more
TL;DR: This paper presents the implementation of MapReduce, a programming model and an associated implementation for processing and generating large data sets that runs on a large cluster of commodity machines and is highly scalable.
Proceedings Article
Resilient distributed datasets: a fault-tolerant abstraction for in-memory cluster computing
Matei Zaharia,Mosharaf Chowdhury,Tathagata Das,Ankur Dave,Justin Ma,Murphy McCauley,Michael J. Franklin,Scott Shenker,Ion Stoica +8 more
TL;DR: Resilient Distributed Datasets is presented, a distributed memory abstraction that lets programmers perform in-memory computations on large clusters in a fault-tolerant manner and is implemented in a system called Spark, which is evaluated through a variety of user applications and benchmarks.
Journal ArticleDOI
A bridging model for parallel computation
TL;DR: The bulk-synchronous parallel (BSP) model is introduced as a candidate for this role, and results quantifying its efficiency both in implementing high-level language features and algorithms, as well as in being implemented in hardware.
Proceedings ArticleDOI
Pregel: a system for large-scale graph processing
Grzegorz Malewicz,Matthew H. Austern,Aart J. C. Bik,James C. Dehnert,Ilan Horn,Naty Leiser,Grzegorz Czajkowski +6 more
TL;DR: A model for processing large graphs that has been designed for efficient, scalable and fault-tolerant implementation on clusters of thousands of commodity computers, and its implied synchronicity makes reasoning about programs easier.
Proceedings ArticleDOI
Dryad: distributed data-parallel programs from sequential building blocks
TL;DR: The Dryad execution engine handles all the difficult problems of creating a large distributed, concurrent application: scheduling the use of computers and their CPUs, recovering from communication or computer failures, and transporting data between vertices.