Showing papers on "Sparse grid published in 2016"

Understanding machine‐learned density functionals

Abstract: Machine learning (ML) is an increasingly popular statistical tool for analyzing either measured or calculated data sets. Here, we explore its application to a well-defined physics problem, investigating issues of how the underlying physics is handled by ML, and how self-consistent solutions can be found by limiting the domain in which ML is applied. The particular problem is how to find accurate approximate density functionals for the kinetic energy (KE) of noninteracting electrons. Kernel ridge regression is used to approximate the KE of non-interacting fermions in a one dimensional box as a functional of their density. The properties of different kernels and methods of cross-validation are explored, reproducing the physics faithfully in some cases, but not others. We also address how self-consistency can be achieved with information on only a limited electronic density domain. Accurate constrained optimal densities are found via a modified Euler-Lagrange constrained minimization of the machine-learned total energy, despite the poor quality of its functional derivative. A projected gradient descent algorithm is derived using local principal component analysis. Additionally, a sparse grid representation of the density can be used without degrading the performance of the methods. The implications for machine-learned density functional approximations are discussed. © 2015 Wiley Periodicals, Inc.

Dimension-Adaptive Sparse Grid Interpolation for Uncertainty Quantification in Modern Power Systems: Probabilistic Power Flow

Abstract: In this paper, the authors firstly present the theoretical foundation of a state-of-the-art uncertainty quantification method, the dimension-adaptive sparse grid interpolation (DASGI), for introducing it into the applications of probabilistic power flow (PPF), specifically as discussed herein. It is well-known that numerous sources of uncertainty are being brought into the present-day electrical grid, by large-scale integration of renewable, thus volatile, generation, e.g., wind power, and by unprecedented load behaviors. In presence of these added uncertainties, it is imperative to change traditional deterministic power flow (DPF) calculation to take them into account in the routine operation and planning. However, the PPF analysis is still quite challenging due to two features of the uncertainty in modern power systems: high dimensionality and presence of stochastic interdependence. Both are traditionally addressed by the Monte Carlo simulation (MCS) at the cost of cumbersome computation; in this paper instead, they are tackled with the joint application of the DASGI and Copula theory (especially advantageous for constructing nonlinear dependence among various uncertainty sources), in order to accomplish the dependent high-dimensional PPF analysis in an accurate and faster manner. Based on the theory of DASGI, its combination with Copula and the DPF for the PPF is also introduced systematically in this work. Finally, the feasibility and the effectiveness of this methodology are validated by the test results of two standard IEEE test cases.

Using Adaptive Sparse Grids to Solve High-Dimensional Dynamic Models

Abstract: We present a flexible and scalable method for computing global solutions of high-dimensional stochastic dynamic models. Within a time iteration or value function iteration setup, we interpolate functions using an adaptive sparse grid algorithm. With increasing dimensions, sparse grids grow much more slowly than standard tensor product grids. Moreover, adaptivity adds a second layer of sparsity, as grid points are added only where they are most needed, for instance in regions with steep gradients or at non-differentiabilities. To further speed up the solution process, our implementation is fully hybrid parallel, combining distributed and shared memory parallelization paradigms, and thus allows for an efficient use of high-performance computing architectures. To demonstrate the broad applicability of our method, we apply it to two very different dynamic models: First, to high-dimensional international real business cycle models with capital adjustment costs and irreversible investment. Second, to multi-good menu-cost models with temporary sales and economies of scope in price setting.

Convergence of quasi-optimal sparse-grid approximation of Hilbert-space-valued functions: application to random elliptic PDEs

Abstract: In this work we provide a convergence analysis for the quasi-optimal version of the sparse-grids stochastic collocation method we presented in a previous work: "On the optimal polynomial approximation of stochastic PDEs by Galerkin and collocation methods" (Beck et al., Math Models Methods Appl Sci 22(09), 2012). The construction of a sparse grid is recast into a knapsack problem: a profit is assigned to each hierarchical surplus and only the most profitable ones are added to the sparse grid. The convergence rate of the sparse grid approximation error with respect to the number of points in the grid is then shown to depend on weighted summability properties of the sequence of profits. This is a very general argument that can be applied to sparse grids built with any uni-variate family of points, both nested and non-nested. As an example, we apply such quasi-optimal sparse grids to the solution of a particular elliptic PDE with stochastic diffusion coefficients, namely the "inclusions problem": we detail the convergence estimates obtained in this case using polynomial interpolation on either nested (Clenshaw---Curtis) or non-nested (Gauss---Legendre) abscissas, verify their sharpness numerically, and compare the performance of the resulting quasi-optimal grids with a few alternative sparse-grid construction schemes recently proposed in the literature.

An Adaptive Sparse Grid Algorithm for Elliptic PDEs with Lognormal Diffusion Coefficient

Abstract: In this work we build on the classical adaptive sparse grid algorithm (T. Gerstner and M. Griebel, Dimension-adaptive tensor-product quadrature), obtaining an enhanced version capable of using non-nested collocation points, and supporting quadrature and interpolation on unbounded sets. We also consider several profit indicators that are suitable to drive the adaptation process. We then use such algorithm to solve an important test case in Uncertainty Quantification problem, namely the Darcy equation with lognormal permeability random field, and compare the results with those obtained with the quasi-optimal sparse grids based on profit estimates, which we have proposed in our previous works (cf. e.g. Convergence of quasi-optimal sparse grids approximation of Hilbert-valued functions: application to random elliptic PDEs). To treat the case of rough permeability fields, in which a sparse grid approach may not be suitable, we propose to use the adaptive sparse grid quadrature as a control variate in a Monte Carlo simulation. Numerical results show that the adaptive sparse grids have performances similar to those of the quasi-optimal sparse grids and are very effective in the case of smooth permeability fields. Moreover, their use as control variate in a Monte Carlo simulation allows to tackle efficiently also problems with rough coefficients, significantly improving the performances of a standard Monte Carlo scheme.

A dynamically adaptive sparse grids method for quasi-optimal interpolation of multidimensional functions

Abstract: In this work we develop a dynamically adaptive sparse grids (SG) method for quasi-optimal interpolation of multidimensional analytic functions defined over a product of one dimensional bounded domains. The goal of such approach is to construct an interpolant in space that corresponds to the "best M -terms" based on sharp a priori estimate of polynomial coefficients. In the past, SG methods have been successful in achieving this, with a traditional construction that relies on the solution to a Knapsack problem: only the most profitable hierarchical surpluses are added to the SG. However, this approach requires additional sharp estimates related to the size of the analytic region and the norm of the interpolation operator, i.e., the Lebesgue constant. Instead, we present an iterative SG procedure that adaptively refines an estimate of the region and accounts for the effects of the Lebesgue constant. Our approach does not require any a priori knowledge of the analyticity or operator norm, is easily generalized to both affine and non-affine analytic functions, and can be applied to sparse grids built from one dimensional rules with arbitrary growth of the number of nodes. In several numerical examples, we utilize our dynamically adaptive SG to interpolate quantities of interest related to the solutions of parametrized elliptic and hyperbolic PDEs, and compare the performance of our quasi-optimal interpolant to several alternative SG schemes.

A Sparse Grid Discontinuous Galerkin Method for High-Dimensional Transport Equations and Its Application to Kinetic Simulations

Abstract: In this paper, we develop a sparse grid discontinuous Galerkin (DG) scheme for transport equations and apply it to kinetic simulations. The method uses the weak formulations of traditional Runge--Kutta DG (RKDG) schemes for hyperbolic problems and is proved to be $L^2$ stable and convergent. A major advantage of the scheme lies in its low computational and storage cost due to the employed sparse finite element approximation space. This attractive feature is explored in simulating Vlasov and Boltzmann transport equations. Good performance in accuracy and conservation is verified by numerical tests in up to four dimensions.

Induction Motor Parameter Estimation Using Sparse Grid Optimization Algorithm

Abstract: Inaccurate motor parameters can lead to an inefficient motor control. Although several motor estimation methods have been utilized to estimate motor parameters, it is still challenging to ensure a good level of confidence in the estimation. In this paper, we propose a novel offline induction motor parameter estimation method based on sparse grid optimization algorithm. The estimation is achieved by matching the response of machines mathematical model with recorded stator current and voltage signals. This approach is noninvasive as it uses external measurements, resulting in reduced system complexity and cost. A globally optimal point was found by sampling on the sparse grid, which was created using the hyperbolic cross points and additional heuristics. This has resulted in reducing the total number of search points, and provided the best match between the mathematical model and measurement data. The estimated motor parameters can be further refined by using any local search method. The experimental results indicate a very good agreement between estimated values and reference values.

Sparse Pseudo Spectral Projection Methods with Directional Adaptation for Uncertainty Quantification

Abstract: We investigate two methods to build a polynomial approximation of a model output depending on some parameters. The two approaches are based on pseudo-spectral projection (PSP) methods on adaptively constructed sparse grids, and aim at providing a finer control of the resolution along two distinct subsets of model parameters. The control of the error along different subsets of parameters may be needed for instance in the case of a model depending on uncertain parameters and deterministic design variables. We first consider a nested approach where an independent adaptive sparse grid PSP is performed along the first set of directions only, and at each point a sparse grid is constructed adaptively in the second set of directions. We then consider the application of aPSP in the space of all parameters, and introduce directional refinement criteria to provide a tighter control of the projection error along individual dimensions. Specifically, we use a Sobol decomposition of the projection surpluses to tune the sparse grid adaptation. The behavior and performance of the two approaches are compared for a simple two-dimensional test problem and for a shock-tube ignition model involving 22 uncertain parameters and 3 design parameters. The numerical experiments indicate that whereas both methods provide effective means for tuning the quality of the representation along distinct subsets of parameters, PSP in the global parameter space generally requires fewer model evaluations than the nested approach to achieve similar projection error. In addition, the global approach is better suited for generalization to more than two subsets of directions.

A sparse grid discontinuous Galerkin method for high-dimensional transport equations and its application to kinetic simulations

Abstract: In this paper, we develop a sparse grid discontinuous Galerkin (DG) scheme for transport equations and applied it to kinetic simulations. The method uses the weak formulations of traditional Runge-Kutta DG (RKDG) schemes for hyperbolic problems and is proven to be $L^2$ stable and convergent. A major advantage of the scheme lies in its low computational and storage cost due to the employed sparse finite element approximation space. This attractive feature is explored in simulating Vlasov and Boltzmann transport equations. Good performance in accuracy and conservation is verified by numerical tests in up to four dimensions.

Adaptive Sparse Grid Model Order Reduction for Fast Bayesian Estimation and Inversion

Abstract: We present new sparse-grid based algorithms for fast Bayesian estimation and inversion of parametric operator equations. We propose Reduced Basis (RB) acceleration of numerical integration based on Smolyak sparse grid quadrature. To tackle the curse-of-dimensionality in high-dimensional Bayesian inversion, we exploit sparsity of the parametric forward solution map as well as of the Bayesian posterior density with respect to the random parameters. We employ an dimension adaptive Sparse Grid method (aSG) for both, offline-training the reduced basis as well as for deterministic quadrature of the conditional expectations which arise in Bayesian estimates. For the forward problem with nonaffine dependence on the random variables, we perform further affine approximation based on the Empirical Interpolation Method (EIM) proposed in [1]. A novel combined algorithm to adaptively refine the sparse grid used for quadrature approximation of the Bayesian estimates, of the reduced basis approximation and to compress the parametric forward solutions by empirical interpolation is proposed. The theoretically predicted computational efficiency which is independent of the number of active parameters is demonstrated in numerical experiments for a model, nonaffine-parametric, stationary, elliptic diffusion problem, in two spacial and in parameter space dimensions up to 1024.

Sparse Grids for the Vlasov–Poisson Equation

Abstract: The Vlasov–Poisson equation models the evolution of a plasma in an external or self-consistent electric field. The model consists of an advection equation in six dimensional phase space coupled to Poisson’s equation. Due to the high dimensionality and the development of small structures the numerical solution is quite challenging. For two or four dimensional Vlasov problems, semi-Lagrangian solvers have been successfully applied. Introducing a sparse grid, the number of grid points can be reduced in higher dimensions. In this paper, we present a semi-Lagrangian Vlasov–Poisson solver on a tensor product of two sparse grids. In order to defeat the problem of poor representation of Gaussians on the sparse grid, we introduce a multiplicative delta-f method and separate a Gaussian part that is then handled analytically. In the semi-Lagrangian setting, we have to evaluate the hierarchical surplus on each mesh point. This interpolation step is quite expensive on a sparse grid due to the global nature of the basis functions. In our method, we use an operator splitting so that the advection steps boil down to a number of one dimensional interpolation problems. With this structure in mind we devise an evaluation algorithm with constant instead of logarithmic complexity per grid point. Results are shown for standard test cases and in four dimensional phase space the results are compared to a full-grid solution and a solution on the four dimensional sparse grid.

Complex scientific applications made fault-tolerant with the sparse grid combination technique:

Abstract: Ultra-large–scale simulations via solving partial differential equations (PDEs) require very large computational systems for their timely solution. Studies shown the rate of failure grows with the system size, and these trends are likely to worsen in future machines. Thus, as systems, and the problems solved on them, continue to grow, the ability to survive failures is becoming a critical aspect of algorithm development. The sparse grid combination technique (SGCT) which is a cost-effective method for solving higher dimensional PDEs can be easily modified to provide algorithm-based fault tolerance.In this article, we describe how the SGCT can produce fault-tolerant versions of the Gyrokinetic Electromagnetic Numerical Experiment plasma application, Taxila Lattice Boltzmann Method application, and Solid Fuel Ignition application. We use an alternate component grid combination formula by adding some redundancy on the SGCT to recover data from lost processes. User-level failure mitigation (ULFM) message pass...

Hyperspherical Sparse Approximation Techniques for High-Dimensional Discontinuity Detection

Abstract: This work proposes a hyperspherical sparse approximation framework for detecting jump discontinuities in functions in high-dimensional spaces. The need for a novel approach results from the theoretical and computational inefficiencies of well-known approaches, such as adaptive sparse grids, for discontinuity detection. Our approach constructs the hyperspherical coordinate representation of the discontinuity surface of a function. Then sparse approximations of the transformed function are built in the hyperspherical coordinate system, with values at each point estimated by solving a one-dimensional discontinuity detection problem. Due to the smoothness of the hypersurface, the new technique can identify jump discontinuities with significantly reduced computational cost, compared to existing methods. Several approaches are used to approximate the transformed discontinuity surface in the hyperspherical system, including adaptive sparse grid and radial basis function interpolation, discrete least squares proj...

Sparse grid techniques for particle-in-cell schemes

Abstract: We propose the use of sparse grids to accelerate particle-in-cell (PIC) schemes. By using the so-called `combination technique' from the sparse grids literature, we are able to dramatically increase the size of the spatial cells in multi-dimensional PIC schemes while paying only a slight penalty in grid-based error. The resulting increase in cell size allows us to reduce the statistical noise in the simulation without increasing total particle number. We present initial proof-of-principle results from test cases in two and three dimensions that demonstrate the new scheme's efficiency, both in terms of computation time and memory usage.

Efficient spectral sparse grid approximations for solving multi-dimensional forward backward SDEs

Abstract: This is the second part in a series of papers on multi-step schemes for solving coupled forward backward stochastic differential equations (FBSDEs). We extend the basic idea in our former paper [W. Zhao, Y. Fu and T. Zhou, SIAM J. Sci. Comput., 36 (2014), pp. A1731-A1751] to solve high-dimensional FBSDEs, by using the spectral sparse grid approximations. The main issue for solving high dimensional FBSDEs is to build an efficient spatial discretization, and deal with the related high dimensional conditional expectations and interpolations. In this work, we propose the sparse grid spatial discretization. We use the sparse grid Gaussian-Hermite quadrature rule to approximate the conditional expectations. And for the associated high dimensional interpolations, we adopt an spectral expansion of functions in polynomial spaces with respect to the spatial variables, and use the sparse grid approximations to recover the expansion coefficients. The FFT algorithm is used to speed up the recovery procedure, and the entire algorithm admits efficient and high accurate approximations in high-dimensions, provided that the solutions are sufficiently smooth. Several numerical examples are presented to demonstrate the efficiency of the proposed methods.

Convergence of Sparse Collocation for Functions of Countably Many Gaussian Random Variables (with Application to Elliptic PDEs)

Abstract: We give a convergence proof for the approximation by sparse collocation of Hilbert-space-valued functions depending on countably many Gaussian random variables. Such functions appear as solutions of elliptic PDEs with lognormal diffusion coefficients. We outline a general $L^2$-convergence theory based on previous work by Bachmayr et al. (2016) and Chen (2016) and establish an algebraic convergence rate for sufficiently smooth functions assuming a mild growth bound for the univariate hierarchical surpluses of the interpolation scheme applied to Hermite polynomials. We verify specifically for Gauss-Hermite nodes that this assumption holds and also show algebraic convergence w.r.t. the resulting number of sparse grid points for this case. Numerical experiments illustrate the dimension-independent convergence rate.

Krylov Integration Factor Method on Sparse Grids for High Spatial Dimension Convection---Diffusion Equations

Abstract: Krylov implicit integration factor (IIF) methods were developed in Chen and Zhang (J Comput Phys 230:4336---4352, 2011) for solving stiff reaction---diffusion equations on high dimensional unstructured meshes. The methods were further extended to solve stiff advection---diffusion---reaction equations in Jiang and Zhang (J Comput Phys 253:368---388, 2013). Recently we studied the computational power of Krylov subspace approximations on dealing with high dimensional problems. It was shown that the Krylov integration factor methods have linear computational complexity and are especially efficient for high dimensional convection---diffusion problems with anisotropic diffusions. In this paper, we combine the Krylov integration factor methods with sparse grid combination techniques and solve high spatial dimension convection---diffusion equations such as Fokker---Planck equations on sparse grids. Numerical examples are presented to show that significant computational times are saved by applying the Krylov integration factor methods on sparse grids.

Hierarchical Gradient-Based Optimization with B-Splines on Sparse Grids

Abstract: Optimization algorithms typically perform a series of function evaluations to find an approximation of an optimal point of the objective function. Evaluations can be expensive, e.g., if they depend on the results of a complex simulation. When dealing with higher-dimensional functions, the curse of dimensionality increases the difficulty of the problem rapidly and prohibits a regular sampling. Instead of directly optimizing the objective function, we replace it with a sparse grid interpolant, saving valuable function evaluations. We generalize the standard piecewise linear basis to hierarchical B-splines, making the sparse grid surrogate smooth enough to enable gradient-based optimization methods. Also, we use an uncommon refinement criterion due to Novak and Ritter to generate an appropriate sparse grid adaptively. Finally, we evaluate the new method for various artificial and real-world examples.

Sparse Grid Surrogate Models for Electromagnetic Problems With Many Parameters

Abstract: A sparse grid surrogate model (or metamodel) is proposed to reduce the computational time involved by accurate electromagnetic (EM) simulators. Sparse grids have already been used in many applications for interpolation and integration. The method can treat a high ber of independent parameters that are intractable for many other techniques due to the curse of dimensionality. Beyond the conventional method, adaptive sparse girds are also studied. The capabilities are illustrated through the examples drawn from the EM nondestructive evaluation.

Enabling accurate first-principle calculations of electronic properties with a corrected k.p scheme

Abstract: A computationally inexpensive k.p-based interpolation scheme is developed that can extend the eigenvalues and momentum matrix elements of a sparsely sampled k-point grid into a densely sampled one. Dense sampling, often required to accurately describe transport and optical properties of bulk materials, can be demanding to compute, for instance, in combination with hybrid functionals in density functional theory (DFT) or with perturbative expansions beyond DFT such as the $GW$ method. The scheme is based on solving the k.p method and extrapolating from multiple reference k-points. It includes a correction term that reduces the number of empty bands needed and ameliorates band discontinuities. We show how the scheme can be used to generate accurate band structures, density of states, and dielectric functions. Several examples are given, using traditional and hybrid functionals, with Si, TiNiSn, and Cu as model materials. We illustrate that d-electron and semi-core states, which are particular challenging for the k.p method, can be handled with the correction scheme if the sparse grid is not too sparse.