The FEniCS Project is a collaborative project for the development of innovative concepts and tools for automated scientific computing, with a particular focus on the solution of differential equations by finite element methods. The FEniCS Projects software consists of a collection of interoperable software components, including DOLFIN, FFC, FIAT, Instant, UFC, UFL, and mshr. This note describes the new features and changes introduced in the release of FEniCS version 1.5.

https://publications.lib.chalmers.se/records/fulltext/228672/local_228672.pdf

The FEniCS Project Version 1.5

We consider the problem of designing a dynamic scheduling strategy that takes into account both workload and memory information in the context of the parallel multifrontal factorization. The originality of our approach is that we base our estimations (work and memory) on a static optimistic scenario during the analysis phase. This scenario is then used during the factorization phase to constrain the dynamic decisions that compute fully irregular partitions in order to better balance the workload. We show that our new scheduling algorithm significantly improves both the memory behaviour and the factorization time. We give experimental results for large challenging real-life 3D problems on 64 and 128 processors.

/pdf/hybrid-scheduling-for-the-parallel-solution-of-linear-7h62u6cen9.pdf

Hybrid scheduling for the parallel solution of linear systems

CHOLMOD is a set of routines for factorizing sparse symmetric positive definite matrices of the form A or AAT, updating/downdating a sparse Cholesky factorization, solving linear systems, updating/downdating the solution to the triangular system Lx = b, and many other sparse matrix functions for both symmetric and unsymmetric matrices. Its supernodal Cholesky factorization relies on LAPACK and the Level-3 BLAS, and obtains a substantial fraction of the peak performance of the BLAS. Both real and complex matrices are supported. CHOLMOD is written in ANSI/ISO C, with both C and MATLABTM interfaces. It appears in MATLAB 7.2 as x = A\b when A is sparse symmetric positive definite, as well as in several other sparse matrix functions.

/pdf/algorithm-887-cholmod-supernodal-sparse-cholesky-15p7kbqjy7.pdf

Algorithm 887: CHOLMOD, Supernodal Sparse Cholesky Factorization and Update/Downdate

The parallel ordering of large graphs is a difficult problem, because on the one hand minimum degree algorithms do not parallelize well, and on the other hand the obtainment of high quality orderings with the nested dissection algorithm requires efficient graph bipartitioning heuristics, the best sequential implementations of which are also hard to parallelize. This paper presents a set of algorithms, implemented in the PT-Scotch software package, which allows one to order large graphs in parallel, yielding orderings the quality of which is only slightly worse than the one of state-of-the-art sequential algorithms. Our implementation uses the classical nested dissection approach but relies on several novel features to solve the parallel graph bipartitioning problem. Thanks to these improvements, PT-Scotch produces consistently better orderings than ParMeTiS on large numbers of processors.

/pdf/pt-scotch-a-tool-for-efficient-parallel-graph-ordering-3nl2pt7ac8.pdf

PT-Scotch: A tool for efficient parallel graph ordering

We present forward and adjoint spectral-element simulations of coupled acoustic and (an)elastic seismic wave propagation on fully unstructured hexahedral meshes. Simulations benefit from recent advances in hexahedral meshing, load balancing and software optimization. Meshing may be accomplished using a mesh generation tool kit such as CUBIT, and load balancing is facilitated by graph partitioning based on the SCOTCH library. Coupling between fluid and solid regions is incorporated in a straightforward fashion using domain decomposition. Topography, bathymetry and Moho undulations may be readily included in the mesh, and physical dispersion and attenuation associated with anelasticity are accounted for using a series of standard linear solids. Finite-frequency Fr'echet derivatives are calculated using adjoint methods in both fluid and solid domains. The software is benchmarked for a layercake model. We present various examples of fully unstructured meshes, snapshots of wavefields and finite-frequency kernels generated by Version 2.0 'Sesame' of our widely used open source spectral-element package SPECFEM3D.

/pdf/forward-and-adjoint-simulations-of-seismic-wave-propagation-133ae34l3b.pdf

Forward and adjoint simulations of seismic wave propagation on fully unstructured hexahedral meshes

PaStiX: A High-Performance Parallel Direct Solver for Sparse Symmetric Definite Systems

We present finite-element numerical simulations of seismic wave propagation in non linear inelastic geological media. We demonstrate the feasibility of large-scale modeling based on an implicit numerical scheme and a nonlinear constitutive model. We illustrate our methodology with an application to regional scale modeling in the French Riviera, which is prone to earthquakes. The PaStiX direct solver is used to handle large matrix numerical factorizations based on hybrid parallelism to reduce memory overhead. A specific methodology is introduced for the parallel assembly in the context of soil nonlinearity. We analyse the scaling of the parallel algorithms on large-scale configurations and we discuss the physical results.

/pdf/high-performance-finite-element-simulations-of-seismic-wave-1uyryny4vd.pdf

High-performance finite-element simulations of seismic wave propagation in three-dimensional nonlinear inelastic geological media

Parallel computing is widely used for large scale three-dimensional simulation of seismic wave propagation. One particularity of most of these simulations is to consider a finite computing domain whereas the physical problem is unbounded. Additional numerical conditions are then required to absorb the energy at the artificial boundaries, which introduces a different formulation and a load-imbalance. In the context of finite difference method, we study the use of thread overloading approach to alleviate the imbalance. We introduce a mixed-hybrid parallel implementation based on a classical cartesian partitioning at the MPI level and a self-scheduling algorithm at the thread level to handle more than 700 threads on 8 processors. We demonstrate the efficiency of our methodology on an example of regional modeling performed on 80 processors.

Exploiting Intensive Multithreading for the Efficient Simulation of 3D Seismic Wave Propagation

In this paper, we present the developments realized in the O uragan project around the parallelization of a M atlab -like tool called S cilab . These developments use high-performance numerical libraries and different approaches based either on the duplication of S cilab processes or on computational servers. This tool, S cilab //, allows users to perform high-level operations on distributed matrices in a metacomputing environment. We also present performance results on different architectures.

https://hal.inria.fr/inria-00072419/document

SCILAB to SCILAB // : the Ouragan project

Since the last decade, most of the supercomputer architectures are based on clusters of SMP nodes. In those architectures the exchanges between processors are made through shared memory when the processors are located on a same SMP node and through the network otherwise. Generally, the MPI implementations provided by the constructor on those machines are adapted to this situation and take advantage of the shared memory to treat messages between processors in a same SMP node. Nevertheless, this transparent approach to exploit shared memory does not avoid the storage of the extra-structures needed to manage efficiently the communications between processors. For high performance parallel direct solvers, the storage of these extra-structures can become a bottleneck. In this paper, we propose an hybrid MPI-thread implementation of a parallel direct solver and analyse the benefits of this approach in terms of memory and run-time performances.

Jean Roman

Papers

PaStiX: A High-Performance Parallel Direct Solver for Sparse Symmetric Definite Systems

High-performance finite-element simulations of seismic wave propagation in three-dimensional nonlinear inelastic geological media

Exploiting Intensive Multithreading for the Efficient Simulation of 3D Seismic Wave Propagation

SCILAB to SCILAB // : the Ouragan project

On using an hybrid MPI-Thread programming for the implementation of a parallel sparse direct solver on a network of SMP nodes