Open AccessProceedings Article
On Load Shedding in Complex Event Processing
Yeye He,Siddharth Barman,Jeffrey F. Naughton +2 more
- pp 213-224
TLDR
This paper formalizes broad classes of CEP load-shedding scenarios as different optimization problems and demonstrates an array of complexity results that reveal the hardness of these problems and construct shedding algorithms with performance guarantees.Abstract:
Complex Event Processing (CEP) is a stream processing model that focuses on detecting event patterns in continuous event streams. While the CEP model has gained popularity in the research communities and commercial technologies, the problem of gracefully degrading performance under heavy load in the presence of resource constraints, or load shedding, has been largely overlooked. CEP is similar to “classical” stream data management, but addresses a substantially different class of queries. This unfortunately renders the load shedding algorithms developed for stream data processing inapplicable. In this paper we study CEP load shedding under various resource constraints. We formalize broad classes of CEP load-shedding scenarios as different optimization problems. We demonstrate an array of complexity results that reveal the hardness of these problems and construct shedding algorithms with performance guarantees. Our results shed some light on the difficulty of developing load-shedding algorithms that maximize utility.read more
Citations
More filters
Journal ArticleDOI
When things matter
Yongrui Qin,Quan Z. Sheng,Nickolas Falkner,Schahram Dustdar,Hua Wang,Athanasios V. Vasilakos +5 more
TL;DR: The main techniques and state-of-the-art research efforts in IoT from data-centric perspectives are reviewed, including data stream processing, data storage models, complex event processing, and searching in IoT.
Posted Content
When Things Matter: A Data-Centric View of the Internet of Things
Yongrui Qin,Quan Z. Sheng,Nickolas Falkner,Schahram Dustdar,Hua Wang,Athanasios V. Vasilakos +5 more
TL;DR: The main techniques and state-of-the-art research efforts in IoT from data-centric perspectives are surveyed, including data stream processing, data storage models, complex event processing, and searching in IoT.
Proceedings ArticleDOI
Load-aware shedding in stream processing systems
TL;DR: This paper provides a theoretical analysis proving that LAS is an (ε, δ)-approximation of the optimal online load shedder and shows its performance through a practical evaluation based both on simulations and on a running prototype.
Journal ArticleDOI
Microblogs data management: a survey
TL;DR: This paper reviews core components that enable large-scale querying and indexing for microblogs data, and discusses system-level issues and on-going effort on supporting microblogs through the rising wave of big data systems.
Proceedings ArticleDOI
Load Shedding for Complex Event Processing: Input-based and State-based Techniques
TL;DR: This work introduces a hybrid model that combines both input-based and statebased shedding to achieve high result quality under constrained resources and indicates that such hybrid shedding improves the recall by up to 14× for synthetic data and 11.4× for real-world data, compared to baseline approaches.
References
More filters
Proceedings ArticleDOI
A Load Shedding Framework and Optimizations for M-way Windowed Stream Joins
TL;DR: This paper introduces GrubJoin; an adaptive, m-way, windowed stream join that effectively performs time correlation-aware CPU load shedding, and formalizes window harvesting as an optimization problem, develop greedy heuristics to determine near-optimal window harvesting configurations and use approximation techniques to capture the time correlations.
Book ChapterDOI
On approximation algorithms for concave quadratic programming
TL;DR: It is argued that such an approximation can be found in polynomial time for fixed $\epsilon$ and $k$, where $k$ denotes the number of negative eigenvalues.
Proceedings ArticleDOI
Efficient Construction of Compact Shedding Filters for Data Stream Processing
TL;DR: This paper structure NR sifters as a set of (sifter region, drop threshold) pairs to achieve compactness, develop query consolidation techniques to enable quick construction, and introduce flexible space partitioning mechanisms to realize fast lookup.
Proceedings ArticleDOI
Inter-domain stealthy port scan detection through complex event processing
TL;DR: An algorithm for the detection of interdomain SYN port scans named Rank-based SYN (R-SYN) port scan detection algorithm which combines and adapts three detection techniques in order to obtain a unique global statement about the malicious behavior of host activities.
Proceedings ArticleDOI
Supporting Generic Cost Models for Wide-Area Stream Processing
TL;DR: XFlow is presented, a generic data stream collection, processing, and dissemination system that addresses this limitation efficiently and can effectively optimize various performance metrics in the presence of varying network and workload conditions.