Despite great progress in simulating multiphysics problems using the numerical discretization of partial differential equations (PDEs), one still cannot seamlessly incorporate noisy data into existing algorithms, mesh generation remains complex, and high-dimensional problems governed by parameterized PDEs cannot be tackled. Moreover, solving inverse problems with hidden physics is often prohibitively expensive and requires different formulations and elaborate computer codes. Machine learning has emerged as a promising alternative, but training deep neural networks requires big data, not always available for scientific problems. Instead, such networks can be trained from additional information obtained by enforcing the physical laws (for example, at random points in the continuous space-time domain). Such physics-informed learning integrates (noisy) data and mathematical models, and implements them through neural networks or other kernel-based regression networks. Moreover, it may be possible to design specialized network architectures that automatically satisfy some of the physical invariants for better accuracy, faster training and improved generalization. Here, we review some of the prevailing trends in embedding physics into machine learning, present some of the current capabilities and limitations and discuss diverse applications of physics-informed learning both for forward and inverse problems, including discovering hidden physics and tackling high-dimensional problems. The rapidly developing field of physics-informed learning integrates data and mathematical models seamlessly, enabling accurate inference of realistic and high-dimensional multiphysics problems. This Review discusses the methodology and provides diverse examples and an outlook for further developments.

Physics-informed machine learning

Fluid dynamic turbulence is one of the most challenging computational physics problems because of the extremely wide range of time and space scales involved, the strong nonlinearity of the governing equations, and the many practical and important applications. While most linear fluid instabilities are well understood, the nonlinear interactions among them makes even the relatively simple limit of homogeneous isotropic turbulence difficult to treat physically, mathematically, and computationally. Turbulence is modeled computationally by a two-stage bootstrap process. The first stage, direct numerical simulation, attempts to resolve the relevant physical time and space scales but its application is limited to diffusive flows with a relatively small Reynolds number (Re). Using direct numerical simulation to provide a database, in turn, allows calibration of phenomenological turbulence models for engineering applications. Large eddy simulation incorporates a form of turbulence modeling applicable when the large-scale flows of interest are intrinsically time dependent, thus throwing common statistical models into question. A promising approach to large eddy simulation involves the use of high-resolution monotone computational fluid dynamics algorithms such as flux-corrected transport or the piecewise parabolic method which have intrinsic subgrid turbulence models coupled naturally to the resolved scales in the computed flow. The physical considerations underlying and evidence supporting this monotone integrated large eddy simulation approach are discussed.

New insights into large eddy simulation

It is widely known that neural networks (NNs) are universal approximators of continuous functions. However, a less known but powerful result is that a NN with a single hidden layer can accurately approximate any nonlinear continuous operator. This universal approximation theorem of operators is suggestive of the structure and potential of deep neural networks (DNNs) in learning continuous operators or complex systems from streams of scattered data. Here, we thus extend this theorem to DNNs. We design a new network with small generalization error, the deep operator network (DeepONet), which consists of a DNN for encoding the discrete input function space (branch net) and another DNN for encoding the domain of the output functions (trunk net). We demonstrate that DeepONet can learn various explicit operators, such as integrals and fractional Laplacians, as well as implicit operators that represent deterministic and stochastic differential equations. We study different formulations of the input function space and its effect on the generalization error for 16 different diverse applications. Neural networks are known as universal approximators of continuous functions, but they can also approximate any mathematical operator (mapping a function to another function), which is an important capability for complex systems such as robotics control. A new deep neural network called DeepONet can lean various mathematical operators with small generalization error.

Learning nonlinear operators via DeepONet based on the universal approximation theorem of operators

Adaptive activation functions accelerate convergence in deep and physics-informed neural networks

Surrogate modeling for fluid flows based on physics-constrained deep learning without simulation data

Vortex-induced vibrations of bluff bodies occur when the vortex shedding frequency is close to the natural frequency of the structure. Of interest is the prediction of the lift and drag forces on the structure given some limited and scattered information on the velocity field. This is an inverse problem that is not straightforward to solve using standard computational fluid dynamics methods, especially since no information is provided for the pressure. An even greater challenge is to infer the lift and drag forces given some dye or smoke visualizations of the flow field. Here we employ deep neural networks that are extended to encode the incompressible Navier–Stokes equations coupled with the structure’s dynamic motion equation. In the first case, given scattered data in space–time on the velocity field and the structure’s motion, we use four coupled deep neural networks to infer very accurately the structural parameters, the entire time-dependent pressure field (with no prior training data), and reconstruct the velocity vector field and the structure’s dynamic motion. In the second case, given scattered data in space–time on a concentration field only, we use five coupled deep neural networks to infer very accurately the vector velocity field and all other quantities of interest as before. This new paradigm of inference in fluid mechanics for coupled multi-physics problems enables velocity and pressure quantification from flow snapshots in small subdomains and can be exploited for flow control applications and also for system identification.

Deep learning of vortex-induced vibrations

\n Physics-informed neural networks (PINNs) have gained popularity across different engineering fields due to their effectiveness in solving realistic problems with noisy data and often partially missing physics. In PINNs, automatic differentiation is leveraged to evaluate differential operators without discretization errors, and a multitask learning problem is defined in order to simultaneously fit observed data while respecting the underlying governing laws of physics. Here, we present applications of PINNs to various prototype heat transfer problems, targeting in particular realistic conditions not readily tackled with traditional computational methods. To this end, we first consider forced and mixed convection with unknown thermal boundary conditions on the heated surfaces and aim to obtain the temperature and velocity fields everywhere in the domain, including the boundaries, given some sparse temperature measurements. We also consider the prototype Stefan problem for two-phase flow, aiming to infer the moving interface, the velocity and temperature fields everywhere as well as the different conductivities of a solid and a liquid phase, given a few temperature measurements inside the domain. Finally, we present some realistic industrial applications related to power electronics to highlight the practicality of PINNs as well as the effective use of neural networks in solving general heat transfer problems of industrial complexity. Taken together, the results presented herein demonstrate that PINNs not only can solve ill-posed problems, which are beyond the reach of traditional computational methods, but they can also bridge the gap between computational and experimental heat transfer.

Physics-Informed Neural Networks for Heat Transfer Problems

DeepM&Mnet: Inferring the electroconvection multiphysics fields based on operator approximation by neural networks

We have demonstrated the effectiveness of reinforcement learning (RL) in bluff body flow control problems both in experiments and simulations by automatically discovering active control strategies for drag reduction in turbulent flow. Specifically, we aimed to maximize the power gain efficiency by properly selecting the rotational speed of two small cylinders, located parallel to and downstream of the main cylinder. By properly defining rewards and designing noise reduction techniques, and after an automatic sequence of tens of towing experiments, the RL agent was shown to discover a control strategy that is comparable to the optimal strategy found through lengthy systematically planned control experiments. Subsequently, these results were verified by simulations that enabled us to gain insight into the physical mechanisms of the drag reduction process. While RL has been used effectively previously in idealized computer flow simulation studies, this study demonstrates its effectiveness in experimental fluid mechanics and verifies it by simulations, potentially paving the way for efficient exploration of additional active flow control strategies in other complex fluid mechanics applications.

https://www.pnas.org/content/pnas/117/42/26091.full.pdf

Reinforcement learning for bluff body active flow control in experiments and simulations.

Vortex induced vibrations of bluff bodies occur when the vortex shedding frequency is close to the natural frequency of the structure Of interest is the prediction of the lift and drag forces on the structure given some limited and scattered information on the velocity field This is an inverse problem that is not straightforward to solve using standard computational fluid dynamics (CFD) methods, especially since no information is provided for the pressure An even greater challenge is to infer the lift and drag forces given some dye or smoke visualizations of the flow field Here we employ deep neural networks that are extended to encode the incompressible Navier-Stokes equations coupled with the structure's dynamic motion equation In the first case, given scattered data in space-time on the velocity field and the structure's motion, we use four coupled deep neural networks to infer very accurately the structural parameters, the entire time-dependent pressure field (with no prior training data), and reconstruct the velocity vector field and the structure's dynamic motion In the second case, given scattered data in space-time on a concentration field only, we use five coupled deep neural networks to infer very accurately the vector velocity field and all other quantities of interest as before This new paradigm of inference in fluid mechanics for coupled multi-physics problems enables velocity and pressure quantification from flow snapshots in small subdomains and can be exploited for flow control applications and also for system identification

Zhicheng Wang

Papers

Deep learning of vortex-induced vibrations

Physics-Informed Neural Networks for Heat Transfer Problems

DeepM&Mnet: Inferring the electroconvection multiphysics fields based on operator approximation by neural networks

Reinforcement learning for bluff body active flow control in experiments and simulations.

Deep Learning of Vortex Induced Vibrations