Showing papers on "Discretization published in 2018"

fPINNs: Fractional Physics-Informed Neural Networks

Abstract: Physics-informed neural networks (PINNs) are effective in solving integer-order partial differential equations (PDEs) based on scattered and noisy data. PINNs employ standard feedforward neural networks (NNs) with the PDEs explicitly encoded into the NN using automatic differentiation, while the sum of the mean-squared PDE-residuals and the mean-squared error in initial/boundary conditions is minimized with respect to the NN parameters. We extend PINNs to fractional PINNs (fPINNs) to solve space-time fractional advection-diffusion equations (fractional ADEs), and we demonstrate their accuracy and effectiveness in solving multi-dimensional forward and inverse problems with forcing terms whose values are only known at randomly scattered spatio-temporal coordinates (black-box forcing terms). A novel element of the fPINNs is the hybrid approach that we introduce for constructing the residual in the loss function using both automatic differentiation for the integer-order operators and numerical discretization for the fractional operators. We consider 1D time-dependent fractional ADEs and compare white-box (WB) and black-box (BB) forcing. We observe that for the BB forcing fPINNs outperform FDM. Subsequently, we consider multi-dimensional time-, space-, and space-time-fractional ADEs using the directional fractional Laplacian and we observe relative errors of $10^{-4}$. Finally, we solve several inverse problems in 1D, 2D, and 3D to identify the fractional orders, diffusion coefficients, and transport velocities and obtain accurate results even in the presence of significant noise.

Discrete Cosserat Approach for Multisection Soft Manipulator Dynamics

Abstract: Nowadays, the most adopted model for the design and control of soft robots is the piecewise constant curvature model, with its consolidated benefits and drawbacks. In this work, an alternative model for multisection soft manipulator dynamics is presented based on a discrete Cosserat approach, in which the continuous Cosserat model is discretized by assuming a piecewise constant strain along the soft arm. As a consequence, the soft manipulator state is described by a finite set of constant strains. This approach has several advantages with respect to the existing models. First, it takes into account shear and torsional deformations, which are both essential to cope with out-of-plane external loads. Furthermore, it inherits desirable geometrical and mechanical properties of the continuous Cosserat model, such as intrinsic parameterization and greater generality. Finally, this approach allows to extend to soft manipulators, the recursive composite-rigid-body and articulated-body algorithms, whose performances are compared through a cantilever beam simulation. The soundness of the model is demonstrated through extensive simulation and experimental results.

A moving least squares material point method with displacement discontinuity and two-way rigid body coupling

Abstract: In this paper, we introduce the Moving Least Squares Material Point Method (MLS-MPM). MLS-MPM naturally leads to the formulation of Affine Particle-In-Cell (APIC) [Jiang et al. 2015] and Polynomial Particle-In-Cell [Fu et al. 2017] in a way that is consistent with a Galerkin-style weak form discretization of the governing equations. Additionally, it enables a new stress divergence discretization that effortlessly allows all MPM simulations to run two times faster than before. We also develop a Compatible Particle-In-Cell (CPIC) algorithm on top of MLS-MPM. Utilizing a colored distance field representation and a novel compatibility condition for particles and grid nodes, our framework enables the simulation of various new phenomena that are not previously supported by MPM, including material cutting, dynamic open boundaries, and two-way coupling with rigid bodies. MLS-MPM with CPIC is easy to implement and friendly to performance optimization.

Numerical methods for fractional diffusion

Abstract: We present three schemes for the numerical approximation of fractional diffusion, which build on different definitions of such a non-local process. The first method is a PDE approach that applies to the spectral definition and exploits the extension to one higher dimension. The second method is the integral formulation and deals with singular non-integrable kernels. The third method is a discretization of the Dunford–Taylor formula. We discuss pros and cons of each method, error estimates, and document their performance with a few numerical experiments.

Forward-Backward Stochastic Neural Networks: Deep Learning of High-dimensional Partial Differential Equations

Abstract: Classical numerical methods for solving partial differential equations suffer from the curse dimensionality mainly due to their reliance on meticulously generated spatio-temporal grids. Inspired by modern deep learning based techniques for solving forward and inverse problems associated with partial differential equations, we circumvent the tyranny of numerical discretization by devising an algorithm that is scalable to high-dimensions. In particular, we approximate the unknown solution by a deep neural network which essentially enables us to benefit from the merits of automatic differentiation. To train the aforementioned neural network we leverage the well-known connection between high-dimensional partial differential equations and forward-backward stochastic differential equations. In fact, independent realizations of a standard Brownian motion will act as training data. We test the effectiveness of our approach for a couple of benchmark problems spanning a number of scientific domains including Black-Scholes-Barenblatt and Hamilton-Jacobi-Bellman equations, both in 100-dimensions.

The fully nonconforming virtual element method for biharmonic problems

Abstract: In this paper, we address the numerical approximation of linear fourth-order elliptic problems on polygonal meshes. In particular, we present a novel nonconforming virtual element discretization of arbitrary order of accuracy for biharmonic problems. The approximation space is made of possibly discontinuous functions, thus giving rise to the fully nonconforming virtual element method. We derive optimal error estimates in a suitable (broken) energy norm and present numerical results to assess the validity of the theoretical estimates.

A New Representation in PSO for Discretization-Based Feature Selection

Abstract: In machine learning, discretization and feature selection (FS) are important techniques for preprocessing data to improve the performance of an algorithm on high-dimensional data. Since many FS methods require discrete data, a common practice is to apply discretization before FS. In addition, for the sake of efficiency, features are usually discretized individually (or univariate). This scheme works based on the assumption that each feature independently influences the task, which may not hold in cases where feature interactions exist. Therefore, univariate discretization may degrade the performance of the FS stage since information showing feature interactions may be lost during the discretization process. Initial results of our previous proposed method [evolve particle swarm optimization (EPSO)] showed that combining discretization and FS in a single stage using bare-bones particle swarm optimization (BBPSO) can lead to a better performance than applying them in two separate stages. In this paper, we propose a new method called potential particle swarm optimization (PPSO) which employs a new representation that can reduce the search space of the problem and a new fitness function to better evaluate candidate solutions to guide the search. The results on ten high-dimensional datasets show that PPSO select less than 5% of the number of features for all datasets. Compared with the two-stage approach which uses BBPSO for FS on the discretized data, PPSO achieves significantly higher accuracy on seven datasets. In addition, PPSO obtains better (or similar) classification performance than EPSO on eight datasets with a smaller number of selected features on six datasets. Furthermore, PPSO also outperforms the three compared (traditional) methods and performs similar to one method on most datasets in terms of both generalization ability and learning capacity.

Retarded Potentials and Time Domain Boundary Integral Equations: A Road Map

Abstract: The retarted layer potentials.- From time domain to Laplace domain.- From Laplace domain to time domain.- Convulution Quadrature.- The Discrete layer potentials.- A General Class of Second Order Differential Equations.- Time domain analysis of the single layer potential.- Time domain analysis of the double layer potential.- Full discretization revisited.- Patterns, Extensions, and Conclusions.- Appendices.

A boundary-type meshless solver for transient heat conduction analysis of slender functionally graded materials with exponential variations

Abstract: This study presents a parallel meshless solver for transient heat conduction analysis of slender functionally graded materials (FGMs) with exponential variations. In the present parallel meshless solver, a strong-form boundary collocation method, the boundary knot method (BKM), in conjunction with Laplace transform is implemented to solve the heat conduction equations of slender FGMs with exponential variations. This method is mathematically simple, easy-to-parallel, meshless, and without domain discretization. However, two ill-posed issues, the ill-conditioning dense BKM matrix and numerical inverse Laplace transform process, may lead to incorrect numerical results. Here the extended precision arithmetic (EPA) and the domain decomposition method (DDM) have been adopted to alleviate the effect of these two ill-posed issues on numerical efficiency of the present method. Then the parallel algorithm has been employed to significantly reduce the computational cost and enhance the computational capacity for the FGM structures with larger length-width ratio. To demonstrate the effectiveness of the present parallel meshless solver for transient heat conduction analysis, several benchmark examples are considered under slender FGMs with exponential variations. The present results are compared with the analytical solutions, the conventional boundary knot method and COMSOL simulation.

Robust Discretization of Flow in Fractured Porous Media

Abstract: Flow in fractured porous media represents a challenge for discretization methods due to the disparate scales and complex geometry. Herein we propose a new discretization, based on the mixed finite ...

The RKHS method for numerical treatment for integrodifferential algebraic systems of temporal two-point BVPs

Abstract: Many problems arising in different fields of sciences and engineering can be reduced, by applying some appropriate discretization, either to a system of integrodifferential algebraic equations or to a sequence of such systems. The aim of the present analysis is to implement a relatively recent computational method, reproducing kernel Hilbert space, for obtaining the solutions of integrodifferential algebraic systems of temporal two-point boundary value problems. Two extended inner product spaces W[0, 1] and H[0, 1] are constructed in which the boundary conditions of the systems are satisfied, while two smooth kernel functions R t (s) and r t (s) are used throughout the evolution of the algorithm in order to obtain the required grid points. An efficient construction is given to obtain the numerical solutions for the systems together with an existence proof of the exact solutions based upon the reproducing kernel theory. In this approach, computational results of some numerical examples are presented to illustrate the viability, simplicity, and applicability of the algorithm developed.

pyomo.dae : a modeling and automatic discretization framework for optimization with differential and algebraic equations

Abstract: We describe pyomo.dae, an open source Python-based modeling framework that enables high-level abstract specification of optimization problems with differential and algebraic equations. The pyomo.dae framework is integrated with the Pyomo open source algebraic modeling language, and is available at http://www.pyomo.org . One key feature of pyomo.dae is that it does not restrict users to standard, predefined forms of differential equations, providing a high degree of modeling flexibility and the ability to express constraints that cannot be easily specified in other modeling frameworks. Other key features of pyomo.dae are the ability to specify optimization problems with high-order differential equations and partial differential equations, defined on restricted domain types, and the ability to automatically transform high-level abstract models into finite-dimensional algebraic problems that can be solved with off-the-shelf solvers. Moreover, pyomo.dae users can leverage existing capabilities of Pyomo to embed differential equation models within stochastic and integer programming models and mathematical programs with equilibrium constraint formulations. Collectively, these features enable the exploration of new modeling concepts, discretization schemes, and the benchmarking of state-of-the-art optimization solvers.

The BR1 Scheme is Stable for the Compressible Navier–Stokes Equations

Abstract: In this work we prove that the original (Bassi and Rebay in J Comput Phys 131:267–279, 1997) scheme (BR1) for the discretization of second order viscous terms within the discontinuous Galerkin collocation spectral element method (DGSEM) with Gauss Lobatto nodes is stable. More precisely, we prove in the first part that the BR1 scheme preserves energy stability of the skew-symmetric advection term DGSEM discretization for the linearized compressible Navier–Stokes equations (NSE). In the second part, we prove that the BR1 scheme preserves the entropy stability of the recently developed entropy stable compressible Euler DGSEM discretization of Carpenter et al. (SIAM J Sci Comput 36:B835–B867, 2014) for the non-linear compressible NSE, provided that the auxiliary gradient equations use the entropy variables. Both parts are presented for fully three-dimensional, unstructured curvilinear hexahedral grids. Although the focus of this work is on the BR1 scheme, we show that the proof naturally includes the Local DG scheme of Cockburn and Shu.

A Comparison of Automatic Differentiation and Continuous Sensitivity Analysis for Derivatives of Differential Equation Solutions

Abstract: Derivatives of differential equation solutions are commonly for parameter estimation, fitting neural differential equations, and as model diagnostics. However, with a litany of choices and a Cartesian product of potential methods, it can be difficult for practitioners to understand which method is likely to be the most effective on their particular application. In this manuscript we investigate the performance characteristics of Discrete Local Sensitivity Analysis implemented via Automatic Differentiation (DSAAD) against continuous adjoint sensitivity analysis. Non-stiff and stiff biological and pharmacometric models, including a PDE discretization, are used to quantify the performance of sensitivity analysis methods. Our benchmarks show that on small systems of ODEs (approximately $<100$ parameters+ODEs), forward-mode DSAAD is more efficient than both reverse-mode and continuous forward/adjoint sensitivity analysis. The scalability of continuous adjoint methods is shown to be more efficient than discrete adjoints and forward methods after crossing this size range. These comparative studies demonstrate a trade-off between memory usage and performance in the continuous adjoint methods that should be considered when choosing the technique, while numerically unstable backsolve techniques from the machine learning literature are demonstrated as unsuitable for most scientific models. The performance of adjoint methods is shown to be heavily tied to the reverse-mode AD method, with tape-based AD methods shown to be 2 orders of magnitude slower on nonlinear partial differential equations than static AD techniques. These results also demonstrate the applicability of DSAAD to differential-algebraic equations, delay differential equations, and hybrid differential equation systems, showcasing an ease of implementation advantage for DSAAD approaches.

Direct Runge-Kutta Discretization Achieves Acceleration

Abstract: We study gradient-based optimization methods obtained by directly discretizing a second-order ordinary differential equation (ODE) related to the continuous limit of Nesterov's accelerated gradient method. When the function is smooth enough, we show that acceleration can be achieved by a stable discretization of this ODE using standard Runge-Kutta integrators. Specifically, we prove that under Lipschitz-gradient, convexity and order-$(s+2)$ differentiability assumptions, the sequence of iterates generated by discretizing the proposed second-order ODE converges to the optimal solution at a rate of $\mathcal{O}({N^{-2\frac{s}{s+1}}})$, where $s$ is the order of the Runge-Kutta numerical integrator. Furthermore, we introduce a new local flatness condition on the objective, under which rates even faster than $\mathcal{O}(N^{-2})$ can be achieved with low-order integrators and only gradient information. Notably, this flatness condition is satisfied by several standard loss functions used in machine learning. We provide numerical experiments that verify the theoretical rates predicted by our results.

Wavelets optimization method for evaluation of fractional partial differential equations: an application to financial modelling

Abstract: In the present paper, we employ a wavelets optimization method is employed for the elucidations of fractional partial differential equations of pricing European option accompanied by a Levy model. We apply the Legendre wavelets optimization method (LWOM) to optimize the governing problem. The novelty of the proposed method is the inclusion of differential evolution algorithm (DE) in the Legendre wavelets method for the optimized approximations of the unknown terms of the Legendre wavelets. Sequentially, the functions and components of the pricing models are discretized by utilizing the operational matrix of fractional integration of Legendre wavelets. Illustratively, the implementation of the LWOM is exemplified on a pricing European option Levy model and successfully depicted the stock paths. Moreover, comparison analysis of the Black-Scholes model with a class of Levy model and LWOM with q-homotopy analysis transform method (q-HATM) is also deliberated out. Accordingly, the technique is found to be appropriate for financial models that can be expressed as partial differential equations of integer and fractional orders, subjected to initial or boundary conditions.

Sub-grid modelling for two-dimensional turbulence using neural networks

Abstract: In this investigation, a data-driven turbulence closure framework is introduced and deployed for the sub-grid modelling of Kraichnan turbulence. The novelty of the proposed method lies in the fact that snapshots from high-fidelity numerical data are used to inform artificial neural networks for predicting the turbulence source term through localized grid-resolved information. In particular, our proposed methodology successfully establishes a map between inputs given by stencils of the vorticity and the streamfunction along with information from two well-known eddy-viscosity kernels. Through this we predict the sub-grid vorticity forcing in a temporally and spatially dynamic fashion. Our study is both a-priori and a-posteriori in nature. In the former, we present an extensive hyper-parameter optimization analysis in addition to learning quantification through probability density function based validation of sub-grid predictions. In the latter, we analyse the performance of our framework for flow evolution in a classical decaying two-dimensional turbulence test case in the presence of errors related to temporal and spatial discretization. Statistical assessments in the form of angle-averaged kinetic energy spectra demonstrate the promise of the proposed methodology for sub-grid quantity inference. In addition, it is also observed that some measure of a-posteriori error must be considered during optimal model selection for greater accuracy. The results in this article thus represent a promising development in the formalization of a framework for generation of heuristic-free turbulence closures from data.

Log-concave sampling: Metropolis-Hastings algorithms are fast!

Abstract: We consider the problem of sampling from a strongly log-concave density in $\mathbb{R}^d$, and prove a non-asymptotic upper bound on the mixing time of the Metropolis-adjusted Langevin algorithm (MALA). The method draws samples by simulating a Markov chain obtained from the discretization of an appropriate Langevin diffusion, combined with an accept-reject step. Relative to known guarantees for the unadjusted Langevin algorithm (ULA), our bounds show that the use of an accept-reject step in MALA leads to an exponentially improved dependence on the error-tolerance. Concretely, in order to obtain samples with TV error at most $\delta$ for a density with condition number $\kappa$, we show that MALA requires $\mathcal{O} \big(\kappa d \log(1/\delta) \big)$ steps, as compared to the $\mathcal{O} \big(\kappa^2 d/\delta^2 \big)$ steps established in past work on ULA. We also demonstrate the gains of MALA over ULA for weakly log-concave densities. Furthermore, we derive mixing time bounds for the Metropolized random walk (MRW) and obtain $\mathcal{O}(\kappa)$ mixing time slower than MALA. We provide numerical examples that support our theoretical findings, and demonstrate the benefits of Metropolis-Hastings adjustment for Langevin-type sampling algorithms.