Journal ArticleDOI
Apache Spark: a unified engine for big data processing
Matei Zaharia,Reynold Xin,Patrick Wendell,Tathagata Das,Michael Armbrust,Ankur Dave,Xiangrui Meng,Josh Rosen,Shivaram Venkataraman,Michael J. Franklin,Ali Ghodsi,Joseph E. Gonzalez,Scott Shenker,Ion Stoica +13 more
TLDR
This open source computing framework unifies streaming, batch, and interactive big data workloads to unlock new applications.Abstract:
This open source computing framework unifies streaming, batch, and interactive big data workloads to unlock new applicationsread more
Citations
More filters
Journal ArticleDOI
SpeCH: A scalable framework for data placement of data-intensive services in geo-distributed clouds
TL;DR: This article proposes a scalable and unified framework for data-intensive service data placement into geographically distributed clouds using Spectral Clustering on Hypergraphs, and is therefore called SpeCH, which is effective, efficient, and scalable.
Proceedings Article
OPIEC: An Open Information Extraction Corpus
TL;DR: It is found that most of the facts between entities present in OPIEC cannot be found in DBpedia and/or YAGO, that OIE facts often differ in the level of specificity compared to knowledge base facts, and that Oie open relations are generally highly polysemous.
Journal ArticleDOI
An Edge-Fog-Cloud Architecture of Streaming Analytics for Internet of Things Applications.
Hung Cao,Monica Wachowicz +1 more
TL;DR: A new architecture based on the edge-fog-cloud continuum to analyze IoT data streams for delivering data-driven insights in a smart parking scenario is proposed.
Journal ArticleDOI
Wireless Social Networks: A Survey of Recent Advances, Applications and Challenges
TL;DR: This work provides a comprehensive introduction to social networks and reviews the recent literature on the application of social networks in wireless communications and highlights the potential challenges in communication network design, for a successful implementation of social networking strategies.
Journal ArticleDOI
Machine learning for regional crop yield forecasting in Europe
Dilli Paudel,Hendrik Boogaard,Allard de Wit,Marijn van der Velde,Martin Claverie,Luigi Nisini,Sander Janssen,Sjoukje A. Osinga,Ioannis N. Athanasiadis +8 more
TL;DR: In this paper , the authors proposed a regional machine learning approach for multiple spatial levels based on regional crop yield forecasts from machine learning, which can leverage larger data sizes and capture nonlinear relationships between predictors and yield at regional level.
References
More filters
Journal ArticleDOI
MapReduce: simplified data processing on large clusters
Jeffrey Dean,Sanjay Ghemawat +1 more
TL;DR: This paper presents the implementation of MapReduce, a programming model and an associated implementation for processing and generating large data sets that runs on a large cluster of commodity machines and is highly scalable.
Proceedings Article
Resilient distributed datasets: a fault-tolerant abstraction for in-memory cluster computing
Matei Zaharia,Mosharaf Chowdhury,Tathagata Das,Ankur Dave,Justin Ma,Murphy McCauley,Michael J. Franklin,Scott Shenker,Ion Stoica +8 more
TL;DR: Resilient Distributed Datasets is presented, a distributed memory abstraction that lets programmers perform in-memory computations on large clusters in a fault-tolerant manner and is implemented in a system called Spark, which is evaluated through a variety of user applications and benchmarks.
Journal ArticleDOI
A bridging model for parallel computation
TL;DR: The bulk-synchronous parallel (BSP) model is introduced as a candidate for this role, and results quantifying its efficiency both in implementing high-level language features and algorithms, as well as in being implemented in hardware.
Proceedings ArticleDOI
Pregel: a system for large-scale graph processing
Grzegorz Malewicz,Matthew H. Austern,Aart J. C. Bik,James C. Dehnert,Ilan Horn,Naty Leiser,Grzegorz Czajkowski +6 more
TL;DR: A model for processing large graphs that has been designed for efficient, scalable and fault-tolerant implementation on clusters of thousands of commodity computers, and its implied synchronicity makes reasoning about programs easier.
Proceedings ArticleDOI
Dryad: distributed data-parallel programs from sequential building blocks
TL;DR: The Dryad execution engine handles all the difficult problems of creating a large distributed, concurrent application: scheduling the use of computers and their CPUs, recovering from communication or computer failures, and transporting data between vertices.