Early experiences in using a domain-specific language for large-scale graph analysis

TLDR: The feasibility of using an intuitive Domain-Specific Language (DSL) for graph analysis, using a compiler to translate Green-Marl programs into an equivalent Giraph application, automatically bridging between very different programming models is discussed.

Abstract: Large-scale graph analysis has recently been drawing lots of attention from both industry and academia. Although there are already several frameworks designed for scalable graph analysis, e.g. Giraph [1], all these frameworks adopt non-traditional programming models and APIs. This can significantly lower the productivity of the framework user. This paper discusses the feasibility of using an intuitive Domain-Specific Language (DSL) for graph analysis. Specifically, we use a compiler to translate Green-Marl [5] programs into an equivalent Giraph application, automatically bridging between very different programming models. We observe that the DSL programs are concise and intuitive, and that the compiler generated Giraph implementations exhibit performance on par with that of hand-written ones. However, the DSL compilation cannot but fail if the algorithm is fundamentally not compatible with the target framework. Overall, we believe that the DSL-based approach will provide great productivity benefits once it matures.