Journal ArticleDOI
Apache Spark: a unified engine for big data processing
Matei Zaharia,Reynold Xin,Patrick Wendell,Tathagata Das,Michael Armbrust,Ankur Dave,Xiangrui Meng,Josh Rosen,Shivaram Venkataraman,Michael J. Franklin,Ali Ghodsi,Joseph E. Gonzalez,Scott Shenker,Ion Stoica +13 more
TLDR
This open source computing framework unifies streaming, batch, and interactive big data workloads to unlock new applications.Abstract:
This open source computing framework unifies streaming, batch, and interactive big data workloads to unlock new applicationsread more
Citations
More filters
Journal ArticleDOI
Street Smart in 5G: Vehicular Applications, Communication, and Computing
TL;DR: In this article , the authors provide a comprehensive overview of the state of research on vehicular computing in the emerging age of 5G and big data, highlighting several vehicular applications, investigates their requirements, details the enabling communication technologies and computing paradigms, and studies data analytics pipelines and the integration of these enabling technologies in response to application requirements.
Proceedings ArticleDOI
LineageBA: A Fast, Exact and Scalable Graph Generation for the Barabási-Albert Model
Himchan Park,Min-Soo Kim +1 more
TL;DR: Li et al. as discussed by the authors proposed the concept of lineage relationship for reducing memory usage significantly and the detection of hash collisions for parallelizing the graph generation, which significantly outperforms the state-of-the-art graph generation methods and easily generates 2.5 trillion edges within four hours using a small cluster of PCs.
Journal ArticleDOI
The impact of COVID-19 on the protection of rural traditional village
Li Gang,Wang Fang,Quan Sishi +2 more
TL;DR: This paper analyzes people's preference for Huizhou cultural resources to better realize the more effective and far-reaching development and exploitation of Huiz Zhou cultural resources.
Journal ArticleDOI
Multi-Objective Multi-Learner Robot Trajectory Prediction Method for IoT Mobile Robot Systems
TL;DR: A novel two-stage multi-objective multi-learner model is proposed for robot trajectory prediction that can achieve satisfactory accuracy and robustness for different datasets and is suitable for the accurate prediction of robot trajectory.
Proceedings ArticleDOI
Towards a Multi-engine Query Optimizer for Complex SQL Queries on Big Data
TL;DR: An optimizer is envisioned to facilitate faster distributed SQL analytics over multiple engines, which will perform operator-level optimization using Machine Learning techniques and will exploit the sophisticated data-driven local engine optimizations.
References
More filters
Journal ArticleDOI
MapReduce: simplified data processing on large clusters
Jeffrey Dean,Sanjay Ghemawat +1 more
TL;DR: This paper presents the implementation of MapReduce, a programming model and an associated implementation for processing and generating large data sets that runs on a large cluster of commodity machines and is highly scalable.
Proceedings Article
Resilient distributed datasets: a fault-tolerant abstraction for in-memory cluster computing
Matei Zaharia,Mosharaf Chowdhury,Tathagata Das,Ankur Dave,Justin Ma,Murphy McCauley,Michael J. Franklin,Scott Shenker,Ion Stoica +8 more
TL;DR: Resilient Distributed Datasets is presented, a distributed memory abstraction that lets programmers perform in-memory computations on large clusters in a fault-tolerant manner and is implemented in a system called Spark, which is evaluated through a variety of user applications and benchmarks.
Journal ArticleDOI
A bridging model for parallel computation
TL;DR: The bulk-synchronous parallel (BSP) model is introduced as a candidate for this role, and results quantifying its efficiency both in implementing high-level language features and algorithms, as well as in being implemented in hardware.
Proceedings ArticleDOI
Pregel: a system for large-scale graph processing
Grzegorz Malewicz,Matthew H. Austern,Aart J. C. Bik,James C. Dehnert,Ilan Horn,Naty Leiser,Grzegorz Czajkowski +6 more
TL;DR: A model for processing large graphs that has been designed for efficient, scalable and fault-tolerant implementation on clusters of thousands of commodity computers, and its implied synchronicity makes reasoning about programs easier.
Proceedings ArticleDOI
Dryad: distributed data-parallel programs from sequential building blocks
TL;DR: The Dryad execution engine handles all the difficult problems of creating a large distributed, concurrent application: scheduling the use of computers and their CPUs, recovering from communication or computer failures, and transporting data between vertices.