What is Twitter

Graph is a well known data structure to represent the associated relationships in a variety of applications, e.g., data science and machine learning. Despite a wealth of existing efforts on developing graph processing systems for improving the performance and/or energy efficiency on traditional architectures, dedicated hardware solutions, also referred to as graph processing accelerators, are essential and emerging to provide the benefits significantly beyond what those pure software solutions can offer. In this paper, we conduct a systematical survey regarding the design and implementation of graph processing accelerators. Specifically, we review the relevant techniques in three core components toward a graph processing accelerator: preprocessing, parallel graph computation, and runtime scheduling. We also examine the benchmarks and results in existing studies for evaluating a graph processing accelerator. Interestingly, we find that there is not an absolute winner for all three aspects in graph acceleration due to the diverse characteristics of graph processing and the complexity of hardware configurations. We finally present and discuss several challenges in details, and further explore the opportunities for the future research.

A Survey on Graph Processing Accelerators: Challenges and Opportunities

Today’s GPU graph processing frameworks face scalability and efficiency issues as the graph size exceeds GPU-dedicated memory limit Although recent GPUs can over-subscribe memory with Unified Memory (UM), they incur significant overhead when handling graph-structured data In addition, many popular processing frameworks suffer sub-optimal efficiency due to heavy atomic operations when tracking the active vertices This article presents Grus, a novel system framework that allows GPU graph processing to stay competitive with the ever-growing graph complexity Grus improves space efficiency through a UM trimming scheme tailored to the data access behaviors of graph workloads It also uses a lightweight frontier structure to further reduce atomic operations With easy-to-use interface that abstracts the above details, Grus shows up to 64× average speedup over the state-of-the-art in-memory GPU graph processing framework It allows one to process large graphs of 55 billion edges in seconds with a single GPU

https://dl.acm.org/doi/pdf/10.1145/3444844

Grus: Toward Unified-memory-efficient High-performance Graph Processing on GPU

Graph is a well known data structure to represent the associated relationships in a variety of applications, e.g., data science and machine learning. Despite a wealth of existing efforts on developing graph processing systems for improving the performance and/or energy efficiency on traditional architectures, dedicated hardware solutions, also referred to as graph processing accelerators, are essential and emerging to provide the benefits significantly beyond those pure software solutions can offer. In this paper, we conduct a systematical survey regarding the design and implementation of graph processing accelerator. Specifically, we review the relevant techniques in three core components toward a graph processing accelerator: preprocessing, parallel graph computation and runtime scheduling. We also examine the benchmarks and results in existing studies for evaluating a graph processing accelerator. Interestingly, we find that there is not an absolute winner for all three aspects in graph acceleration due to the diverse characteristics of graph processing and complexity of hardware configurations. We finially present to discuss several challenges in details, and to further explore the opportunities for the future research.

Existing graph processing essentially relies on the underlying iterative execution with synchronous (Sync) and/or asynchronous (Async) engine. Nevertheless, they both suffer from a wide class of inherent serialization arising from data interdependencies within a graph.In this article, we present SymGraph, a judicious graph engine with symbolic iteration that enables the parallelism of dependent computation on vertices. SymGraph allows using abstract symbolic value (instead of the concrete value) for the computation if the desired data is unavailable. To maximize the potential of symbolic iteration, we propose a chain of tailored sophisticated techniques, enabling SymGraph to scale out with a new milestone of efficiency for large-scale graph processing. We evaluate SymGraph in comparison to Sync, Async, and a hybrid of Sync and Async engines. Our results on 12 nodes show that SymGraph outperforms all three graph engines by 1.93x (vs. Sync), 1.98x (vs. Async), and 1.57x (vs. Hybrid) on average. In particular, the performance for PageRank on 32 nodes can be dramatically improved by 16.5x (vs. Sync), 23.3x (vs. Async), and 12.1x (vs. Hybrid), respectively. The efficiency of SymGraph is also validated with at least one order of magnitude improvement in contrast to three specialized graph systems (Naiad, GraphX, and PGX.D).

https://dl.acm.org/doi/pdf/10.1145/3170434

Efficient and Scalable Graph Parallel Processing With Symbolic Execution

We introduce Scaph, a GPU-accelerated graph system that achieves scale-up graph processing on large-scale graphs that are initially partitioned into subgraphs at the host to enable iterative graph computations on the subgraphs on the GPU. For active subgraphs to be processed on GPU at an iteration, the prior work always streams each in its entirety to GPU, even though only the neighboring information for its active vertices will ever be used. In contrast, Scaph boosts performance significantly by reducing the amount of such redundant data transferred, thereby improving the effective utilization of the host-GPU bandwidth drastically. The key novelty of Scaph is to classify adaptively at each iteration whether a subgraph is a high-value subgraph (if it is likely to be traversed extensively in the current and future iterations) or a low-value subgraph (otherwise). Scaph then schedules a sub-graph for graph processing on GPU using two graph processing engines, one for high-value subgraphs, which will be streamed to GPU entirely and iterated over repeatedly, one for low-value subgraphs, for which only the neighboring information needed for its active vertices is transferred. Evaluation on real-world and synthesized large-scale graphs shows that Scaph outperforms the state-of-the-art, Totem (4.12×), Graphie (8.93×), and Garaph (3.71×), on average.

/pdf/scaph-scalable-gpu-accelerated-graph-processing-with-value-1ln8zt8mrr.pdf

Scaph: Scalable GPU-Accelerated Graph Processing with Value-Driven Differential Scheduling

Existing graph processing frameworks greatly improve the performance of memory subsystem, but they are still subject to the underlying modern processor, resulting in the potential inefficiencies for graph processing in the sense of low instruction level parallelism and high branch misprediction. These inefficiencies, in accordance with our comprehensive micro-architectural study, mainly arise out of a wealth of dependencies, serial semantic of instruction streams, and complex conditional instructions in graph processing. In this paper, we propose that a fundamental shift of approach is necessary to break through the inefficiencies of the underlying processor via the dataflow paradigm. It is verified that the idea of applying dataflow approach into graph processing is extremely appealing for the following two reasons. First, as the execution and retirement of instructions only depend on the availability of input data in dataflow model, a high degree of parallelism can be therefore provided to relax the heavy dependency and serial semantic. Second, dataflow is guaranteed to make it possible to reduce the costs of branch misprediction by simultaneously executing all branches of a conditional instruction. Consequently, we make the preliminary attempt to develop the dataflow insight into a specialized graph accelerator. We believe that our work would open a wide range of opportunities to improve the performance of computation and memory access for large-scale graph processing.

Towards Dataflow-Based Graph Accelerator

The invention discloses a graph calculation method suitable for a heterogeneous platform. The method comprises: (1) preprocessing original graph data to obtain a plurality of data blocks and metadatainformation corresponding to each data block; (2) evaluating the calculation density of the current iteration according to the metadata information; (3) if the calculation density is smaller than a preset calculation density threshold value and is not zero, executing a calculation task of current iteration by the host according to the data block; If the calculation density is greater than or equalto the calculation density threshold, the accelerator executes a calculation task of current iteration according to the data block; if the calculation density is 0, graph calculation is completed; (4) updating the state of the node based on the historical state information, and ending the current iteration; and (5) repeatedly executing the steps (2)-(4) until the graph calculation is completed. According to the calculation method, the calculation capability of the heterogeneous platform can be fully utilized, and the overall performance of heterogeneous platform processing graph calculation is improved.

Graph calculation method suitable for heterogeneous platform

Most of existing graph processing systems essentially follow pull-based computation model to handle compute-intensive parts of graph iteration for high parallelism. Considering all vertices and edges are processed in each iteration, pull model may suffers from a large number of invalid (vertex/edge) operations that do not contribute to graph convergence, leading to potential performance degradation. In this paper, we have the insight that these invalid operations can be filtered by leveraging a small fraction of critical information. However, most of critical information are often beyond the visibility of active vertices being processed. We present two novel filtration approaches to (cooperatively) identify out-of-visibility critical information with boundary-cut heuristics and speculative prediction for many graph algorithms. We have integrated both approaches and their hybrid solution into three state-of-art graph processing systems (including Ligra, Gemini, and Polymer). Experimental results using a wide variety of graph algorithms on both real-world and synthetic graph datasets show that neither of these approaches can have an absolute win for all graph algorithms. Boundary-cut, predictive, and hybrid approaches can improve the performance by 115.1, 38.1, and 136.6 percent on average.

Efficient Graph Processing with Invalid Update Filtration

It is commonly agreed that a recommender system based on knowledge graph (KG) should not only use user-item interactions, but also take side information into account to deal with the problem of data sparsity. However, existing KG-based models present unique challenges that have the shortcoming of cold start and redundant iterations. To address the issue, we propose a Feature Fusion-based Knowledge Graph Attention Network (FF-KGAT) for the recommendation. FF-KGAT handles the cold-start problem through introducing user characteristics to fully explore the information of users. Additionally, FF-KGAT introduces the feature fusion graph, which removes slight nodes to obtain fewer iterations. Experimental results show that the proposed model significantly outperforms baseline methods. Particularly, the ndcg is increased by 9.83% and 10.25% on two public datasets, compared with the best performance of existing methods. 

Xianliang Li

Papers

Scaph: Scalable GPU-Accelerated Graph Processing with Value-Driven Differential Scheduling

Towards Dataflow-Based Graph Accelerator

Graph calculation method suitable for heterogeneous platform

Efficient Graph Processing with Invalid Update Filtration

FF-KGAT: Feature Fusion Based Knowledge Graph Attention Network for Recommendation