Open AccessProceedings Article
On Load Shedding in Complex Event Processing
Yeye He,Siddharth Barman,Jeffrey F. Naughton +2 more
- pp 213-224
TLDR
This paper formalizes broad classes of CEP load-shedding scenarios as different optimization problems and demonstrates an array of complexity results that reveal the hardness of these problems and construct shedding algorithms with performance guarantees.Abstract:
Complex Event Processing (CEP) is a stream processing model that focuses on detecting event patterns in continuous event streams. While the CEP model has gained popularity in the research communities and commercial technologies, the problem of gracefully degrading performance under heavy load in the presence of resource constraints, or load shedding, has been largely overlooked. CEP is similar to “classical” stream data management, but addresses a substantially different class of queries. This unfortunately renders the load shedding algorithms developed for stream data processing inapplicable. In this paper we study CEP load shedding under various resource constraints. We formalize broad classes of CEP load-shedding scenarios as different optimization problems. We demonstrate an array of complexity results that reveal the hardness of these problems and construct shedding algorithms with performance guarantees. Our results shed some light on the difficulty of developing load-shedding algorithms that maximize utility.read more
Citations
More filters
Journal ArticleDOI
When things matter
Yongrui Qin,Quan Z. Sheng,Nickolas Falkner,Schahram Dustdar,Hua Wang,Athanasios V. Vasilakos +5 more
TL;DR: The main techniques and state-of-the-art research efforts in IoT from data-centric perspectives are reviewed, including data stream processing, data storage models, complex event processing, and searching in IoT.
Posted Content
When Things Matter: A Data-Centric View of the Internet of Things
Yongrui Qin,Quan Z. Sheng,Nickolas Falkner,Schahram Dustdar,Hua Wang,Athanasios V. Vasilakos +5 more
TL;DR: The main techniques and state-of-the-art research efforts in IoT from data-centric perspectives are surveyed, including data stream processing, data storage models, complex event processing, and searching in IoT.
Proceedings ArticleDOI
Load-aware shedding in stream processing systems
TL;DR: This paper provides a theoretical analysis proving that LAS is an (ε, δ)-approximation of the optimal online load shedder and shows its performance through a practical evaluation based both on simulations and on a running prototype.
Journal ArticleDOI
Microblogs data management: a survey
TL;DR: This paper reviews core components that enable large-scale querying and indexing for microblogs data, and discusses system-level issues and on-going effort on supporting microblogs through the rising wave of big data systems.
Proceedings ArticleDOI
Load Shedding for Complex Event Processing: Input-based and State-based Techniques
TL;DR: This work introduces a hybrid model that combines both input-based and statebased shedding to achieve high result quality under constrained resources and indicates that such hybrid shedding improves the recall by up to 14× for synthetic data and 11.4× for real-world data, compared to baseline approaches.
References
More filters
On the densest k-subgraph problems
U. Feige,M. Seltser +1 more
TL;DR: In this article, the applicability of semidefinite programming for approximating the dense k-subgraph problem was studied and it was shown that the problem remains NP-hard even when the maximum degree in G is three.
Journal ArticleDOI
High-performance dynamic pattern matching over disordered streams
TL;DR: This work proposes clean order-agnostic pattern-detection semantics for AFAs, with new algorithms that allow a very efficient implementation, while retaining significant expressiveness and supporting native handling of out-of-order input, stream revisions, dynamic patterns, and several optimizations.
Journal ArticleDOI
Microsoft CEP server and online behavioral targeting
Mohamed Ali,Ciprian Gerea,Balan Sethu Raman,Beysim Sezgin,Tihomir Tarnavski,Tomer Verona,Ping Wang,Peter Zabback,Asvin Ananthanarayan,A. Kirilov,M. Lu,Alex Raizman,Ramkumar Krishnan,Roman Schindlauer,Torsten Grabs,S. Bjeletich,Badrish Chandramouli,Jonathan Goldstein,S. Bhat,Ying Li,V. Di Nicola,Xiaoyang Sean Wang,David Maier,S. Grell,Olivier Nano,Ivo Jose Garcia Dos Santos +25 more
TL;DR: This demo highlights the features of Microsoft CEP under a workload of web click events and highlights the debugging and supportability tools of the CEP server.
Proceedings ArticleDOI
Sampling algorithms in a stream operator
TL;DR: This paper abstracts the stream sampling process and designs a new stream sample operator that can be used to implement a wide variety of algorithms that perform sampling and sampling-based aggregations within a data stream management system.
Load Shedding Techniques for Data Stream Systems
TL;DR: A systematic approach to load shedding with the objective of maximizing query accuracy has been lacking for data stream monitoring systems processing continuous monitoring queries over data streams.