Journal ArticleDOI
Apache Spark: a unified engine for big data processing
Matei Zaharia,Reynold Xin,Patrick Wendell,Tathagata Das,Michael Armbrust,Ankur Dave,Xiangrui Meng,Josh Rosen,Shivaram Venkataraman,Michael J. Franklin,Ali Ghodsi,Joseph E. Gonzalez,Scott Shenker,Ion Stoica +13 more
Reads0
Chats0
TLDR
This open source computing framework unifies streaming, batch, and interactive big data workloads to unlock new applications.Abstract:
This open source computing framework unifies streaming, batch, and interactive big data workloads to unlock new applicationsread more
Citations
More filters
Journal ArticleDOI
The effects of class rarity on the evaluation of supervised healthcare fraud detection models
TL;DR: It is found that, as expected, fraud detection performance decreased as the fraudulent instances became more rare, and the Train_Test method significantly outperforms Cross-Validation.
Journal ArticleDOI
CIDACS-RL: a novel indexing search and scoring-based record linkage system for huge datasets with high accuracy and scalability.
George Caique Gouveia Barbosa,M Sanni Ali,M Sanni Ali,M Sanni Ali,Bruno Araujo,Sandra Reis,Samila Sena,Maria Yury Ichihara,Julia Moreira Pescarini,Rosemeire L. Fiaccone,Rosemeire L. Fiaccone,Leila Denise Alves Ferreira Amorim,Leila Denise Alves Ferreira Amorim,Robespierre Pita,Marcos Barreto,Marcos Barreto,Marcos Barreto,Liam Smeeth,Mauricio Lima Barreto,Mauricio Lima Barreto +19 more
TL;DR: CIDACS-RL algorithm is an innovative linkage tool for huge datasets, with higher accuracy, improved scalability, and substantially shorter execution time compared to other existing linkage tools.
Journal ArticleDOI
A survey of biodiversity informatics: Concepts, practices, and challenges
Luiz M. R. Gadelha,Pedro C. de Siracusa,Eduardo Dalcin,Luís Alexandre Estevão da Silva,Douglas A. Augusto,Eduardo Krempser,Helen Michelle Affe,Raquel Lopes Costa,Maria Luiza Mondelli,Pedro M. Meirelles,Fabiano L. Thompson,Marcia Chame,Artur Ziviani,Marinez Ferreira de Siqueira +13 more
TL;DR: This article explores each stage of the life cycle of biodiversity data, discussing its methodologies, tools, and challenges.
Journal ArticleDOI
Algorithmic Personalization and the Two-Step Flow of Communication
TL;DR: This study examines the relevance of traditional mass communication’s two-step flow-of-communication theory in relation to algorithmic personalization, arguing that opinion leaders and algorithms both function as gatekeeping agents.
Journal ArticleDOI
Big data technology: developments in current research and emerging landscape
TL;DR: Citation and co-citation analysis reveals that there is cross-functional nature of big data research, which permeates different business sectors and is influenced by themes in engineering and information management.
References
More filters
Journal ArticleDOI
MapReduce: simplified data processing on large clusters
Jeffrey Dean,Sanjay Ghemawat +1 more
TL;DR: This paper presents the implementation of MapReduce, a programming model and an associated implementation for processing and generating large data sets that runs on a large cluster of commodity machines and is highly scalable.
Proceedings Article
Resilient distributed datasets: a fault-tolerant abstraction for in-memory cluster computing
Matei Zaharia,Mosharaf Chowdhury,Tathagata Das,Ankur Dave,Justin Ma,Murphy McCauley,Michael J. Franklin,Scott Shenker,Ion Stoica +8 more
TL;DR: Resilient Distributed Datasets is presented, a distributed memory abstraction that lets programmers perform in-memory computations on large clusters in a fault-tolerant manner and is implemented in a system called Spark, which is evaluated through a variety of user applications and benchmarks.
Journal ArticleDOI
A bridging model for parallel computation
TL;DR: The bulk-synchronous parallel (BSP) model is introduced as a candidate for this role, and results quantifying its efficiency both in implementing high-level language features and algorithms, as well as in being implemented in hardware.
Proceedings ArticleDOI
Pregel: a system for large-scale graph processing
Grzegorz Malewicz,Matthew H. Austern,Aart J. C. Bik,James C. Dehnert,Ilan Horn,Naty Leiser,Grzegorz Czajkowski +6 more
TL;DR: A model for processing large graphs that has been designed for efficient, scalable and fault-tolerant implementation on clusters of thousands of commodity computers, and its implied synchronicity makes reasoning about programs easier.
Proceedings ArticleDOI
Dryad: distributed data-parallel programs from sequential building blocks
TL;DR: The Dryad execution engine handles all the difficult problems of creating a large distributed, concurrent application: scheduling the use of computers and their CPUs, recovering from communication or computer failures, and transporting data between vertices.