In this paper, a Parallel Direct Eigensolver for Sequences of Hermitian Eigenvalue Problems with no tridiagonalization is proposed, denoted by \texttt{PDESHEP}, and it combines direct methods with iterative methods. \texttt{PDESHEP} first reduces a Hermitian matrix to its banded form, then applies a spectrum slicing algorithm to the banded matrix, and finally computes the eigenvectors of the original matrix via backtransform. Therefore, compared with conventional direct eigensolvers, \texttt{PDESHEP} avoids tridiagonalization, which consists of many memory-bounded operations. In this work, the iterative method in \texttt{PDESHEP} is based on the contour integral method implemented in FEAST. The combination of direct methods with iterative methods for banded matrices requires some efficient data redistribution algorithms both from 2D to 1D and from 1D to 2D data structures. Hence, some two-step data redistribution algorithms are proposed, which can be $10\times$ faster than ScaLAPACK routine \texttt{PXGEMR2D}. For the symmetric self-consistent field (SCF) eigenvalue problems, \texttt{PDESHEP} can be on average $1.25\times$ faster than the state-of-the-art direct solver in ELPA when using $4096$ processes. Numerical results are obtained for dense Hermitian matrices from real applications and large real sparse matrices from the SuiteSparse collection.

A Parallel Direct Eigensolver for Sequences of Hermitian Eigenvalue Problems with No Tridiagonalization.

The proteomics data analysis pipeline based on the shotgun method requires efficient data processing methods. The parallel algorithm of mass spectrometry database search faces the problems of rapidly expanding database size but low scalability. The heterogeneous database search algorithm based on deep learning is an effective way to solve this problem. Still, the deep learning-based distributed parallel database search algorithm is lacking. This paper analyzes the database searching computational load using a neural network and designs ReSLB, a workflow for the restricted search of mass spectrometry data based on GPU cluster and neural network scoring algorithm. This work aims to ensure the high scalability of future deep learning-based mass spectrometry distributed databases. In the performance estimation of 256 GPUs, the load imbalance of less than 30% and the parallel efficiency of 60% are achieved. Compared with state-of-the-art, the time cost is reduced by 75%, and the parallel efficiency from 1 to 256 GPUs is 3.6x higher than that.

ReSLB: Load Balanced Workflow for Distributed Deep Learning Mass Spectrometry Database

/pdf/editorial-for-the-special-issue-on-architecture-algorithms-2b4our1p.pdf

Editorial for the special issue on architecture, algorithms and applications of high performance sparse matrix computations

In this paper, we propose some efficient data redistribution algorithms for redistributing matrices from 1D or 2D irregular format to block cyclic data distribution (BCDD) format, which can be much faster than the BLACS routine \texttt{PXGEMR2D}. These algorithms can be used to combine direct methods with iterative methods. The proposed algorithms divide the communication into two phases: one for processes in the same column and the other for processes in the same row, and the whole data redistribution task is divided into several independent sub-communications. The communication time can be reduced a lot compared with BLACS. Performance results show that our algorithms can be $2\times$--$5\times$ faster than the BLACS routine \texttt{PXGEMR2D} when using $4096$ processes and the experiments are performed on Tianhe-2A supercomputer.

Efficient data redistribution algorithms from irregular to block cyclic data distribution

A parallel structured banded DC algorithm for symmetric eigenvalue problems

Novel artificial intelligence (AI) technology has expedited various scientific research, e.g., cosmology, physics and bioinformatics, inevitably becoming a significant category of work-load on high performance computing (HPC) systems. Existing AI benchmarks tend to customize well-recognized AI applications, so as to evaluate the AI performance of HPC systems under predefined problem size, in terms of datasets and AI models. However, driven by novel AI technology, most of AI applications are evolving fast on models and datasets to achieve higher accuracy and be applicable to more scenarios. Due to lack of scalability on the problem size, static AI benchmarks might be under competent to help understand the performance trend of evolving AI applications on HPC systems, in particular, the scientific AI applications on large-scale systems. In this paper, we propose a scalable evaluation methodology (SAIH) for analyzing the AI performance trend of HPC systems with scaling the problem sizes of customized AI applications. To enable scalability, SAIH builds a set of novel mechanisms for augmenting problem sizes. As the data and model constantly scale, we can investigate the trend and range of AI performance on HPC systems, and further diagnose system bottlenecks. To verify our methodology, we augment a cosmological AI application to evaluate a real HPC system equipped with GPUs as a case study of SAIH. With data and model augment, SAIH can progressively evaluate the AI performance trend of HPC system, e.g. increasing from 5.2% to 59.6% of the peak theoretical hardware performance. The evaluation results are analyzed and summarized into insight findings on performance issues. For instance, we find that the AI application constantly consumes the I/O bandwidth of shared parallel file system during its iteratively training model. If I/O contention exists, the shared parallel file system might become a bottleneck.

Xia Liao

Papers

Efficient data redistribution algorithms from irregular to block cyclic data distribution

A parallel structured banded DC algorithm for symmetric eigenvalue problems

SAIH: A Scalable Evaluation Methodology for Understanding AI Performance Trend on HPC Systems