Mesay Deleli

Bio: Mesay Deleli is an academic researcher from University of Electronic Science and Technology of China. The author has contributed to research in topics: Graph Edge & Degree (graph theory). The author has an hindex of 1, co-authored 3 publications receiving 1 citations.

OffStreamNG: Partial Stream Hybrid Graph Edge Partitioning Based on Neighborhood Expansion and Greedy Heuristic

Abstract: Recently, graph edge partitioning has shown better partitioning quality than the vertex graph partitioning for the skewed degree distribution of real-world graph data. Graph edge partitioning can be classified as stream and offline. The stream edge partitioning approach supports a big graph partitioning; however, it has lower partitioning quality, is affected by stream order, and it has taken much time to make partitioning compared with the offline edge partitioning. Conversely, the offline edge partitioning approach has better partitioning quality than stream edge partitioning; however, it does not support big graph partitioning. In this study, we propose partial stream hybrid graph edge partitioning OffStreamNG, which leverages the advantage of both offline and stream edge partitioning approaches by interconnecting via saved partition state layer. The OffStreamNG holds vertex and load states as partition state, while the offline component is partitioning using neighborhood expansion heuristic. And it is transferring this partition state to the online component of Greedy heuristic with minor modification of both algorithms. Experimental results show that OffStreamNG achieves attractive results in terms of replication factor, load balance, and total partitioning time.

Taking Heuristic Based Graph Edge Partitioning One Step Ahead via OffStream Partitioning Approach

Abstract: In the modern era of big data, large-scale graph computing has become challenging because of the dramatic rise in graph data size. Graph edge partitioning (GEP) is a crucial preprocessing step to distributed graph platforms, yet it is challenging to partition the large-scale graphs. GEP has shown better partition quality than the graph vertex partitioning for the graph’s skewed degree distribution. Existing GEP approaches are classified into two as stream and offline. The former category assigns edges to the partitions based on the previously received edge information. It has less partitioning quality and is affected by stream order compared to the latter while supporting big graph partitioning. The latter uses complete knowledge of a graph during partitioning and hence has a better partitioning quality than the former; however, it does not support large-scale graphs. In this study, we propose a novel OffStream partitioning approach (OSPA) and hybrid graph edge partitioner OffStreamNH. OSPA leverages both the offline and stream graph partitioning approaches through stateful partitioning by introducing a state layer. This stateful partition state is recorded while offline is partitioning its input graph. It contains partial knowledge of previously partitioned data and is used by the stream partitioner. The OffStreamNH uses Neighborhood Expansion (NE) and Higher Degree Replicated First (HDRF) algorithms for the offline and online; respectively, with minor modifications of both algorithms. Experimental results show that OffStreamNH outperforms the state of the art stream partitioners in terms of replication factor, load balance and tolerates the effect of stream orders.

Graph Computing Systems and Partitioning Techniques: A Survey

Abstract: Graphs are a tremendously suitable data representation that models the relationships of entities in many application domains, such as recommendation systems, machine learning, computational biology, social network analysis, and other application domains. Graphs with many vertices and edges have become quite prevalent in recent years. Therefore, graph computing systems with integrated various graph partitioning techniques have been envisioned as a promising paradigm to handle large-scale graph analytics in these application domains. However, scalable processing of large-scale graphs is challenging due to their high volume and inherent irregular structure of the real-world graphs. Hence, industry and academia have recently proposed graph partitioning and computing systems to efficiently process and analyze large-scale graphs. The graph partitioning and computing systems have been designed to improve scalability issues and reduce processing time complexity. This paper presents an overview, classification, and investigation of the most popular graph partitioning and computing systems. The various methods and approaches of graph partitioning and diverse categories of graph computing systems are presented. Finally, we discuss future challenges and research directions in graph partitioning and computing systems.

Graph Computing Systems and Partitioning Techniques: A Survey

Abstract: Graphs are a tremendously suitable data representations that model the relationships of entities in many application domains, such as recommendation systems, machine learning, computational biology, social network analysis, and other application domains. Graphs with many vertices and edges have become quite prevalent in recent years. Therefore, graph computing systems with integrated various graph partitioning techniques have been envisioned as a promising paradigm to handle large-scale graph analytics in these application domains. However, scalable processing of large-scale graphs is challenging due to their high volume and inherent irregular structure of the real-world graphs. Hence, industry and academia have been recently proposing graph partitioning and computing systems to process and analyze large-scale graphs efficiently. The graph partitioning and computing systems have been designed to improve scalability issues and reduce processing time complexity. This paper presents an overview, classification, and investigation of the most popular graph partitioning and computing systems. The various methods and approaches of graph partitioning and diverse categories of graph computing systems are presented. Finally, we discuss main challenges and future research directions in graph partitioning and computing systems.

Taking Heuristic Based Graph Edge Partitioning One Step Ahead via OffStream Partitioning Approach

Abstract: In the modern era of big data, large-scale graph computing has become challenging because of the dramatic rise in graph data size. Graph edge partitioning (GEP) is a crucial preprocessing step to distributed graph platforms, yet it is challenging to partition the large-scale graphs. GEP has shown better partition quality than the graph vertex partitioning for the graph’s skewed degree distribution. Existing GEP approaches are classified into two as stream and offline. The former category assigns edges to the partitions based on the previously received edge information. It has less partitioning quality and is affected by stream order compared to the latter while supporting big graph partitioning. The latter uses complete knowledge of a graph during partitioning and hence has a better partitioning quality than the former; however, it does not support large-scale graphs. In this study, we propose a novel OffStream partitioning approach (OSPA) and hybrid graph edge partitioner OffStreamNH. OSPA leverages both the offline and stream graph partitioning approaches through stateful partitioning by introducing a state layer. This stateful partition state is recorded while offline is partitioning its input graph. It contains partial knowledge of previously partitioned data and is used by the stream partitioner. The OffStreamNH uses Neighborhood Expansion (NE) and Higher Degree Replicated First (HDRF) algorithms for the offline and online; respectively, with minor modifications of both algorithms. Experimental results show that OffStreamNH outperforms the state of the art stream partitioners in terms of replication factor, load balance and tolerates the effect of stream orders.

Big Data Oriented Graph Division and Storage

Abstract: Parameters of machinery parts have dependency relationships between each other. Given a part, our task is to compute all the combinations of all its parameters constrained by the dependency relationships between them. A directed graph can be used to represent the dependency relationships between those parameters. However, simply using the width-first extension to solve the combinations would lead to the problem of combinatorial explosion, even with the constraint imposed by the dependency relationships. This problem causes consequences of too long the computing time, too large the required computer memory, and too slow the subsequent data querying. To solve the problem, we propose a method of graph division to reduce the computation scale. Based on specific characteristics of our task, the method divides the directed graph into various regions, and further divides complex ones of the regions into various sub-graphs. By the strategy of divide-and-conquer, processing of the entire graph is decomposed into processing of simple sub-graphs. Even with the application of above graph division, rows number of tables used to store the result of the extension can still be as large as one billion. Conventional DBMSs would perform poorly at such large a data scale. To combat this problem, we use a tool of persistent memory programming, Metall, to process the big data. Utilizing the tool gains a high I/O throughout. Our experiments show that, among 3,581 machinery parts being processed, for 97% of them, each part can be processed within 6 hours. Moreover, the technique of graph division improves the efficiency of subsequent data querying by 77%.

Big Data Oriented Graph Division and Storage

Abstract: Parameters of machinery parts have dependency relationships between each other. Given a part, our task is to compute all the combinations of all its parameters constrained by the dependency relationships between them. A directed graph can be used to represent the dependency relationships between those parameters. However, simply using the width-first extension to solve the combinations would lead to the problem of combinatorial explosion, even with the constraint imposed by the dependency relationships. This problem causes consequences of too long the computing time, too large the required computer memory, and too slow the subsequent data querying. To solve the problem, we propose a method of graph division to reduce the computation scale. Based on specific characteristics of our task, the method divides the directed graph into various regions, and further divides complex ones of the regions into various sub-graphs. By the strategy of divide-and-conquer, processing of the entire graph is decomposed into processing of simple sub-graphs. Even with the application of above graph division, rows number of tables used to store the result of the extension can still be as large as one billion. Conventional DBMSs would perform poorly at such large a data scale. To combat this problem, we use a tool of persistent memory programming, Metall, to process the big data. Utilizing the tool gains a high I/O throughout. Our experiments show that, among 3,581 machinery parts being processed, for 97% of them, each part can be processed within 6 hours. Moreover, the technique of graph division improves the efficiency of subsequent data querying by 77%.