Routing algorithms for content-based publish/subscribe systems
TL;DR: This paper presents a compilation of the main algorithms for routing messages in distributed content-based publish-subscribe systems proposed and published in the last decade, matched to similar algorithms familiar to the networking community.
Abstract: In content-based publish/subscribe systems, messages target a dynamic group of participants whose expressed interests match the contents of the messages. In this generalization of multicasting communication, also dubbed content-based networking, naming, binding and communication are intertwined in the same substrate. Optimal content-based routing uses dissemination trees dynamically pruned to only cover the matching subscribers. It is a complex problem that has motivated significant research efforts. This paper presents a compilation of the main algorithms for routing messages in distributed content-based publish-subscribe systems proposed and published in the last decade. Discussion is focused on the content-based routing problem in respect to optimality, complexity and applicability. Moreover, whenever it is appropriate, the algorithms covered are also matched to similar algorithms familiar to the networking community, setting this paper apart from other surveys on the broad topic of publish/subscribe systems.
Citations
More filters
Book•
01 Jan 2006
TL;DR: The authors offer a comprehensive overview, and show the power of event-based architectures in modern system design, encouraging professionals to exploit this technique in next generation large-scale distributed applications like information dissemination, network monitoring, enterprise application integration, or mobile systems.
Abstract: This book provides an in-depth description of event-based systems, covering topics ranging from local event matching and distributed event forwarding algorithms, through a practical discussion of software engineering issues raised by the event-based style, to state-of-the-art research in event-based systems like composite event detection and security. The authors offer a comprehensive overview, and show the power of event-based architectures in modern system design, encouraging professionals to exploit this technique in next generation large-scale distributed applications like information dissemination, network monitoring, enterprise application integration, or mobile systems.
340 citations
TL;DR: A novel four-dimensional (4D) evaluation framework for QoS routing algorithms, whereby the 4D correspond to the type of topology, two forms of scalability of aTopology, and the tightness of the delay constraint, which identifies two algorithms, namely Lagrange relaxation-based aggregated cost (LARAC) and search space reduction delay-cost-constrained routing (SSR+DCCR), that perform very well in most of the4D evaluation space
Abstract: A variety of communication networks, such as industrial communication systems, have to provide strict delay guarantees to the carried flows. Fast and close to optimal quality of service (QoS) routing algorithms, e.g., delay-constrained least-cost (DCLC) routing algorithms, are required for routing flows in such networks with strict delay requirements. The emerging software-defined networking (SDN) paradigm centralizes the network control in SDN controllers that can centrally execute QoS routing algorithms. A wide range of QoS routing algorithms have been proposed in the literature and examined in individual studies. However, a comprehensive evaluation framework and quantitative comparison of QoS routing algorithms that can serve as a basis for selecting and further advancing QoS routing in SDN networks is missing in the literature. This makes it difficult to select the most appropriate QoS routing algorithm for a particular use case, e.g., for SDN controlled industrial communications. We close this gap in the literature by conducting a comprehensive up-to-date survey of centralized QoS routing algorithms. We introduce a novel four-dimensional (4D) evaluation framework for QoS routing algorithms, whereby the 4D correspond to the type of topology, two forms of scalability of a topology, and the tightness of the delay constraint. We implemented 26 selected DCLC algorithms and compared their runtime and cost inefficiency within the 4D evaluation framework. While the main conclusion of this evaluation is that the best algorithm depends on the specific sub-space of the 4D space that is targeted, we identify two algorithms, namely Lagrange relaxation-based aggregated cost (LARAC) and search space reduction delay-cost-constrained routing (SSR+DCCR), that perform very well in most of the 4D evaluation space.
126 citations
Cites background from "Routing algorithms for content-base..."
...and load balancing in the Internet [92], [93], content-based publish/subscribe systems [94], and green routing protocols with sleep scheduling [95]....
[...]
TL;DR: An introduction to the principles of securing event notification, and an analysis of the relevant state-of-the-art by both surveying the academic literature over the period 1998-2014 on secure publish/subscribe services and overviewing the current standards for the marketed products.
Abstract: Publish/subscribe services have encountered considerable success in the building of modern large-scale mission-critical systems. Such systems are characterized by several non-functional requirements, among which security plays a pivotal role due to the emergence of numerous cyber attacks targeting most mission-critical systems. This requires that the adopted publish/subscribe services have to be equipped with the proper means to protect the exchanged data, to preserve their correct behavior and to face possible attack scenarios. Although significant efforts have been made in this field, many issues are still open. This paper includes an introduction to the principles of securing event notification, and an analysis of the relevant state-of-the-art by both surveying the academic literature over the period 1998–2014 on secure publish/subscribe services and overviewing the current standards for the marketed products. Next, it presents the main challenges that are still unresolved and are worthy of further attention in future research efforts.
61 citations
Cites background from "Routing algorithms for content-base..."
...Within the current literature, several possible solutions have been proposed [27]–[29], which are part of the research topic of group communication....
[...]
...Group Communication [29] is the ability of a group of software components or applications to exchange data in a oneto-many or many-to-many fashion....
[...]
TL;DR: The ability of PUB/SUB infrastructures to offer cost-effective, scalable, and quality-aware data distribution in emerging wide-scale and highly dynamic communication environments, such as those related to the continuous exchange of information between static and mobile nodes in smart-city scenarios is focused on.
Abstract: Publish/Subscribe (PUB/SUB) systems have attracted much academic and industrial interest in the last years, with several successful experiences of development and deployment. Notwithstanding this high interest and the relevant research activities accomplished in the field, there are still many open technical challenges calling for additional research efforts. In this paper, we focus on the ability of PUB/SUB infrastructures to offer cost-effective, scalable, and quality-aware data distribution in emerging wide-scale and highly dynamic communication environments, such as those related to the continuous exchange of information between static and mobile nodes in smart-city scenarios. To this purpose, we survey state-of-the-art industrial and academic PUB/SUB solutions, with a strong focus on their support to scalability and quality requirements. We offer a detailed technical analysis of existing mechanisms and techniques for scalable QoS provisioning in PUB/SUB middleware, and we show how different design/implementation details impact the scalability and quality achievable at runtime. At the end of this surveying work, we identify promising guidelines for future research and for PUB/SUB systems extensions to effectively address the technical challenges of scalability and quality.
58 citations
Cites background from "Routing algorithms for content-base..."
..., [1], [5], [15]) have defined and modeled the behavior of PUB/SUB systems....
[...]
...A detailed survey of routing algorithms for PUB/SUB communication can be found in [5] and [9]; in particular, the two papers focus on the sub-field of content-based routing....
[...]
TL;DR: An introduction to the principles of assuring event notification even in the presence of faults, and an analysis of relevant state-of-the-art by both surveying the academic literature over the period 2000-2011 on reliable publish/subscribe services and overviewing current marketed products are covered.
53 citations
Cites background from "Routing algorithms for content-base..."
...Despite there being other surveys published on the broad topic of publish/subscribe services, such as [9,19,20], none of them have focused on the reliability concerns, but they target routing protocols and algorithms to support scalable and efficient event notification....
[...]
References
More filters
TL;DR: A tree is a graph with one and only one path between every two nodes, where at least one path exists between any two nodes and the length of each branch is given.
Abstract: We consider n points (nodes), some or all pairs of which are connected by a branch; the length of each branch is given. We restrict ourselves to the case where at least one path exists between any two nodes. We now consider two problems. Problem 1. Constrnct the tree of minimum total length between the n nodes. (A tree is a graph with one and only one path between every two nodes.) In the course of the construction that we present here, the branches are subdivided into three sets: I. the branches definitely assignec~ to the tree under construction (they will form a subtree) ; II. the branches from which the next branch to be added to set I, will be selected ; III. the remaining branches (rejected or not yet considered). The nodes are subdivided into two sets: A. the nodes connected by the branches of set I, B. the remaining nodes (one and only one branch of set II will lead to each of these nodes), We start the construction by choosing an arbitrary node as the only member of set A, and by placing all branches that end in this node in set II. To start with, set I is empty. From then onwards we perform the following two steps repeatedly. Step 1. The shortest branch of set II is removed from this set and added to
22,704 citations
"Routing algorithms for content-base..." refers background in this paper
...algorithm [29] whenever a notification published by a new node is received....
[...]
TL;DR: The current state of the art of sensor networks is captured in this article, where solutions are discussed under their related protocol stack layer sections.
Abstract: The advancement in wireless communications and electronics has enabled the development of low-cost sensor networks. The sensor networks can be used for various application areas (e.g., health, military, home). For different application areas, there are different technical issues that researchers are currently resolving. The current state of the art of sensor networks is captured in this article, where solutions are discussed under their related protocol stack layer sections. This article also points out the open research issues and intends to spark new interests and developments in this field.
14,048 citations
"Routing algorithms for content-base..." refers background in this paper
...Another scenario where a content-based publish/subscribe model fits well is in wireless sensor networks [6]....
[...]
27 Aug 2001
TL;DR: Results from theoretical analysis, simulations, and experiments show that Chord is scalable, with communication cost and the state maintained by each node scaling logarithmically with the number of Chord nodes.
Abstract: A fundamental problem that confronts peer-to-peer applications is to efficiently locate the node that stores a particular data item. This paper presents Chord, a distributed lookup protocol that addresses this problem. Chord provides support for just one operation: given a key, it maps the key onto a node. Data location can be easily implemented on top of Chord by associating a key with each data item, and storing the key/data item pair at the node to which the key maps. Chord adapts efficiently as nodes join and leave the system, and can answer queries even if the system is continuously changing. Results from theoretical analysis, simulations, and experiments show that Chord is scalable, with communication cost and the state maintained by each node scaling logarithmically with the number of Chord nodes.
10,286 citations
TL;DR: Pastry as mentioned in this paper is a scalable, distributed object location and routing substrate for wide-area peer-to-peer ap- plications, which performs application-level routing and object location in a po- tentially very large overlay network of nodes connected via the Internet.
Abstract: This paper presents the design and evaluation of Pastry, a scalable, distributed object location and routing substrate for wide-area peer-to-peer ap- plications. Pastry performs application-level routing and object location in a po- tentially very large overlay network of nodes connected via the Internet. It can be used to support a variety of peer-to-peer applications, including global data storage, data sharing, group communication and naming. Each node in the Pastry network has a unique identifier (nodeId). When presented with a message and a key, a Pastry node efficiently routes the message to the node with a nodeId that is numerically closest to the key, among all currently live Pastry nodes. Each Pastry node keeps track of its immediate neighbors in the nodeId space, and notifies applications of new node arrivals, node failures and recoveries. Pastry takes into account network locality; it seeks to minimize the distance messages travel, according to a to scalar proximity metric like the number of IP routing hops. Pastry is completely decentralized, scalable, and self-organizing; it automatically adapts to the arrival, departure and failure of nodes. Experimental results obtained with a prototype implementation on an emulated network of up to 100,000 nodes confirm Pastry's scalability and efficiency, its ability to self-organize and adapt to node failures, and its good network locality properties.
7,423 citations
27 Aug 2001
TL;DR: The concept of a Content-Addressable Network (CAN) as a distributed infrastructure that provides hash table-like functionality on Internet-like scales is introduced and its scalability, robustness and low-latency properties are demonstrated through simulation.
Abstract: Hash tables - which map "keys" onto "values" - are an essential building block in modern software systems. We believe a similar functionality would be equally valuable to large distributed systems. In this paper, we introduce the concept of a Content-Addressable Network (CAN) as a distributed infrastructure that provides hash table-like functionality on Internet-like scales. The CAN is scalable, fault-tolerant and completely self-organizing, and we demonstrate its scalability, robustness and low-latency properties through simulation.
6,703 citations