Zhen Zheng

TL;DR: DAPPLE, a synchronous training framework which combines data parallelism and pipeline parallelism for large DNN models, is proposed, which features a novel parallelization strategy planner to solve the partition and placement problems, and explores the optimal hybrid strategies of data and pipeline Parallelism.

TL;DR: An in-depth examination of the state-of-the-art GNN frameworks is provided, revealing five major gaps in the current frameworks in optimizing GNN performance, especially in handling the special complexities of GNN over traditional graph or DNN operations.

TL;DR: DAPPLE as mentioned in this paper is a synchronous training framework which combines data parallelism and pipeline parallelism for large DNN models, and it features a novel parallelization strategy planner to solve the partition and placement problems.

TL;DR: To map the large code base of CAM to the millions of cores on the Sunway system, OpenACC-based refactoring is taken as the major approach, and source-to-source translator tools are applied to exploit the most suitable parallelism for the CPE cluster.

TL;DR: This paper proposes three new execution models equipped with much improved controllability, including a hybrid model that is capable of getting the strengths of all, and leads to the development of a software programming framework named VersaPipe.