Scalable Pattern Sharing on Event Streams
Medhabi Ray,Chuan Lei,Elke A. Rundensteiner +2 more
- pp 495-510
TLDR
The SPASS optimizer identifies opportunities for effective shared processing among CEP queries by leveraging time-based event correlations among queries and finds a shared pattern plan in polynomial-time covering all sequence patterns while still guaranteeing an optimality bound.Abstract:
Complex Event Processing (CEP) has emerged as a technology of choice for high performance event analytics in time-critical decision-making applications. Yet it is becoming increasingly difficult to support high-performance event processing due to the rising number and complexity of event pattern queries and the increasingly high velocity of event streams. In this work we design the SPASS framework that successfully tackles these demanding CEP workloads. Our SPASS optimizer identifies opportunities for effective shared processing among CEP queries by leveraging time-based event correlations among queries. The problem of pattern sharing is shown to be NP-hard by reducing the Minimum Substring Cover problem to our CEP pattern sharing problem. The SPASS optimizer is designed that finds a shared pattern plan in polynomial-time covering all sequence patterns while still guaranteeing an optimality bound. To execute this shared pattern plan, the SPASS runtime employs stream transactions that assure concurrent shared maintenance and re-use of sub-patterns across queries. Our experimental study confirms that the SPASS framework achieves over 16 fold performance improvement for a wide range of experiments compared to the state-of-the-art solution.read more
Citations
More filters
Proceedings ArticleDOI
To Share, or not to Share Online Event Trend Aggregation Over Bursty Event Streams
TL;DR: Hamlet as discussed by the authors proposes a novel framework Hamlet that adaptively decides at run time whether to share or not to share computations depending on the current stream properties to harvest the maximum sharing benefit.
Journal ArticleDOI
Interval-based Queries over Lossy IoT Event Streams
TL;DR: SimpMatch is presented, a novel simplex-based algorithm for probabilistic evaluation of event queries using constraints over event orderings in a stream using the abstraction of segmented intervals and computing the probability of a sequence of such segments using the notion of order statistics.
Proceedings ArticleDOI
hSPICE: state-aware event shedding in complex event processing
TL;DR: In this paper, the authors propose a load shedding approach for complex event processing (CEP) systems that combines these approaches by assigning a utility to an event by considering both the event importance and the importance of PMs.
Proceedings ArticleDOI
Sharon: Shared Online Event Sequence Aggregation
TL;DR: In this paper, a shared online event sequence aggregation (Sharon) approach is proposed to share intermediate aggregates among multiple queries while avoiding the expensive construction of event sequences, which can achieve up to an 18-fold speed-up compared to state-of-the-art approaches.
Journal ArticleDOI
A multiple pattern complex event detection scheme based on decomposition and merge sharing for massive event streams
TL;DR: The experimental results show that the proposed detection method in this article outperforms some general detection methods in detection model and detection algorithm in multiple pattern complex event detection as a whole.
References
More filters
Proceedings Article
TelegraphCQ: Continuous Dataflow Processing for an Uncertain World.
Sirish Chandrasekaran,Owen Cooper,Amol Deshpande,Michael J. Franklin,Joseph M. Hellerstein,Wei Hong,Sailesh Krishnamurthy,Samuel Madden,Vijayshankar Raman,Frederick Reiss,Mehul A. Shah +10 more
TL;DR: The next generation Telegraph system, called TelegraphCQ, is focused on meeting the challenges that arise in handling large streams of continuous queries over high-volume, highly-variable data streams and leverages the PostgreSQL open source code base.
Journal ArticleDOI
NiagaraCQ: a scalable continuous query system for Internet databases
TL;DR: The design of NiagaraCQ system is presented, some experimental results on the system's performance and scalability are given and other techniques including incremental evaluation of continuous queries, use of both pull and push models for detecting heterogeneous data source changes, and memory caching are employed.
Proceedings ArticleDOI
High-performance complex event processing over streams
TL;DR: This paper proposes a complex event language that significantly extends existing event languages to meet the needs of a range of RFID-enabled monitoring applications and describes a query plan-based approach to efficiently implementing this language.
Proceedings ArticleDOI
Continuously adaptive continuous queries over streams
TL;DR: This work presents a continuously adaptive, continuous query (CACQ) implementation based on the eddy query processing framework that provides significant performance benefits over existing approaches to evaluating continuous queries, not only because of its adaptivity, but also because of the aggressive cross-query sharing of work and space that it enables.
Proceedings ArticleDOI
Efficient pattern matching over event streams
TL;DR: This paper presents a formal evaluation model that offers precise semantics for this new class of queries and a query evaluation framework permitting optimizations in a principled way and further analyzes the runtime complexity of query evaluation using this model and develops a suite of techniques that improve runtime efficiency by exploiting sharing in storage and processing.