Extracting governing equations from data is a central challenge in many diverse areas of science and engineering. Data are abundant whereas models often remain elusive, as in climate science, neuroscience, ecology, finance, and epidemiology, to name only a few examples. In this work, we combine sparsity-promoting techniques and machine learning with nonlinear dynamical systems to discover governing equations from noisy measurement data. The only assumption about the structure of the model is that there are only a few important terms that govern the dynamics, so that the equations are sparse in the space of possible functions; this assumption holds for many physical systems in an appropriate basis. In particular, we use sparse regression to determine the fewest terms in the dynamic governing equations required to accurately represent the data. This results in parsimonious models that balance accuracy with model complexity to avoid overfitting. We demonstrate the algorithm on a wide range of problems, from simple canonical systems, including linear and nonlinear oscillators and the chaotic Lorenz system, to the fluid vortex shedding behind an obstacle. The fluid example illustrates the ability of this method to discover the underlying dynamics of a system that took experts in the community nearly 30 years to resolve. We also show that this method generalizes to parameterized systems and systems that are time-varying or have external forcing.

https://www.pnas.org/content/pnas/113/15/3932.full.pdf

Discovering governing equations from data by sparse identification of nonlinear dynamical systems

The Koopman operator is a linear but infinite-dimensional operator that governs the evolution of scalar observables defined on the state space of an autonomous dynamical system and is a powerful tool for the analysis and decomposition of nonlinear dynamical systems In this manuscript, we present a data-driven method for approximating the leading eigenvalues, eigenfunctions, and modes of the Koopman operator The method requires a data set of snapshot pairs and a dictionary of scalar observables, but does not require explicit governing equations or interaction with a “black box” integrator We will show that this approach is, in effect, an extension of dynamic mode decomposition (DMD), which has been used to approximate the Koopman eigenvalues and modes Furthermore, if the data provided to the method are generated by a Markov process instead of a deterministic dynamical system, the algorithm approximates the eigenfunctions of the Kolmogorov backward equation, which could be considered as the “stochastic Koopman operator” (Mezic in Nonlinear Dynamics 41(1–3): 309–325, 2005) Finally, four illustrative examples are presented: two that highlight the quantitative performance of the method when presented with either deterministic or stochastic data and two that show potential applications of the Koopman eigenfunctions

A Data-Driven Approximation of the Koopman Operator: Extending Dynamic Mode Decomposition

Originally introduced in the fluid mechanics community, dynamic mode decomposition (DMD) has emerged as a powerful tool for analyzing the dynamics of nonlinear systems.
 However, existing DMD theory deals primarily with sequential time series for which the measurement dimension is much larger than the number of measurements taken.
 We present a theoretical framework in which we define DMD as the eigendecomposition of an approximating linear operator.
 This generalizes DMD to a larger class of datasets, including nonsequential time series.
 We demonstrate the utility of this approach by presenting novel sampling strategies that increase computational efficiency and mitigate the effects of noise, respectively.
 We also introduce the concept of linear consistency, which helps explain the potential pitfalls of applying DMD to rank-deficient datasets, illustrating with examples.
 Such computations are not considered in the existing literature but can be understood using our more general framework.
 In addition, we show that our theory strengthens the connections between DMD and Koopman operator theory.
 It also establishes connections between DMD and other techniques, including the eigensystem realization algorithm (ERA), a system identification method, and linear inverse modeling (LIM), a method from climate science.
 We show that under certain conditions, DMD is equivalent to LIM.

/pdf/on-dynamic-mode-decomposition-theory-and-applications-pa6gljckm0.pdf

On dynamic mode decomposition: Theory and applications

Simple aerodynamic configurations under even modest conditions can exhibit complex flows with a wide range of temporal and spatial features. It has become common practice in the analysis of these flows to look for and extract physically important features, or modes, as a first step in the analysis. This step typically starts with a modal decomposition of an experimental or numerical dataset of the flowfield, or of an operator relevant to the system. We describe herein some of the dominant techniques for accomplishing these modal decompositions and analyses that have seen a surge of activity in recent decades [1–8]. For a nonexpert, keeping track of recent developments can be daunting, and the intent of this document is to provide an introduction to modal analysis that is accessible to the larger fluid dynamics community. In particular, we present a brief overview of several of the well-established techniques and clearly lay the framework of these methods using familiar linear algebra. The modal analysis techniques covered in this paper include the proper orthogonal decomposition (POD), balanced proper orthogonal decomposition (balanced POD), dynamic mode decomposition (DMD), Koopman analysis, global linear stability analysis, and resolvent analysis.

https://arc.aiaa.org/doi/pdf/10.2514/1.J056060

Modal Analysis of Fluid Flows: An Overview

We propose a sparse regression method capable of discovering the governing partial differential equation(s) of a given system by time series measurements in the spatial domain. The regression framework relies on sparsity-promoting techniques to select the nonlinear and partial derivative terms of the governing equations that most accurately represent the data, bypassing a combinatorially large search through all possible candidate models. The method balances model complexity and regression accuracy by selecting a parsimonious model via Pareto analysis. Time series measurements can be made in an Eulerian framework, where the sensors are fixed spatially, or in a Lagrangian framework, where the sensors move with the dynamics. The method is computationally efficient, robust, and demonstrated to work on a variety of canonical problems spanning a number of scientific domains including Navier-Stokes, the quantum harmonic oscillator, and the diffusion equation. Moreover, the method is capable of disambiguating between potentially nonunique dynamical terms by using multiple time series taken with different initial data. Thus, for a traveling wave, the method can distinguish between a linear wave equation and the Korteweg–de Vries equation, for instance. The method provides a promising new technique for discovering governing equations and physical laws in parameterized spatiotemporal systems, where first-principles derivations are intractable.

Data-driven discovery of partial differential equations.

Originally introduced in the fluid mechanics community, dynamic mode decomposition (DMD) has emerged as a powerful tool for analyzing the dynamics of nonlinear systems. However, existing DMD theory deals primarily with sequential time series for which the measurement dimension is much larger than the number of measurements taken. We present a theoretical framework in which we define DMD as the eigendecomposition of an approximating linear operator. This generalizes DMD to a larger class of datasets, including nonsequential time series. We demonstrate the utility of this approach by presenting novel sampling strategies that increase computational efficiency and mitigate the effects of noise, respectively. We also introduce the concept of linear consistency, which helps explain the potential pitfalls of applying DMD to rank-deficient datasets, illustrating with examples. Such computations are not considered in the existing literature, but can be understood using our more general framework. In addition, we show that our theory strengthens the connections between DMD and Koopman operator theory. It also establishes connections between DMD and other techniques, including the eigensystem realization algorithm (ERA), a system identification method, and linear inverse modeling (LIM), a method from climate science. We show that under certain conditions, DMD is equivalent to LIM.

On Dynamic Mode Decomposition: Theory and Applications

Dynamic mode decomposition (DMD) is an Arnoldi-like method based on the Koopman operator. It analyzes empirical data, typically generated by nonlinear dynamics, and computes eigenvalues and eigenmodes of an approximate linear model. Without explicit knowledge of the dynamical operator, it extracts frequencies, growth rates, and spatial structures for each mode. We show that expansion in DMD modes is unique under certain conditions. When constructing mode-based reduced-order models of partial differential equations, subtracting a mean from the data set is typically necessary to satisfy boundary conditions. Subtracting the mean of the data exactly reduces DMD to the temporal discrete Fourier transform (DFT); this is restrictive and generally undesirable. On the other hand, subtracting an equilibrium point generally preserves the DMD spectrum and modes. Next, we introduce an “optimized” DMD that computes an arbitrary number of dynamical modes from a data set. Compared to DMD, optimized DMD is superior at calculating physically relevant frequencies, and is less numerically sensitive. We test these decomposition methods on data from a two-dimensional cylinder fluid flow at a Reynolds number of 60. Time-varying modes computed from the DMD variants yield low projection errors.

http://cwrowley.princeton.edu/papers/ChenDMD10.pdf

Variants of Dynamic Mode Decomposition: Boundary Condition, Koopman, and Fourier Analyses

This work develops compressed sensing strategies for computing the dynamic mode decomposition (DMD) from heavily subsampled or compressed data.
The resulting DMD eigenvalues are equal to DMD eigenvalues from the full-state data.
It is then possible to reconstruct full-state DMD eigenvectors using $\ell_1$-minimization or greedy algorithms.
If full-state snapshots are available, it may be computationally beneficial to compress the data, compute DMD on the compressed data, and then reconstruct full-state modes by applying the compressed DMD transforms to full-state snapshots.
  
These results rely on a number of theoretical advances. First, we establish connections between DMD on full-state and compressed data. Next, we demonstrate the invariance of the DMD algorithm to left and right unitary transformations.
When data and modes are sparse in some transform basis, we show a similar invariance of DMD to measurement matrices that satisfy the restricted isometry property from compressed sensing.
We demonstrate the success of this architecture on two model systems.
In the first example, we construct a spatial signal from a sparse vector of Fourier coefficients with a linear dynamical system driving the coefficients.
In the second example, we consider the double gyre flow field, which is a model for chaotic mixing in the ocean.
 A video abstract of this work may be found at: http://youtu.be/4tLSq_PEFms .

Compressed sensing and dynamic mode decomposition

The low-frequency unsteadiness in the direct numerical simulation of a Mach 2.9 shock wave/turbulent boundary layer interaction with mean flow separation is analysed using dynamic mode decomposition (DMD). The analysis is applied both to three-dimensional and spanwise-averaged snapshots of the flow. The observed low-frequency DMD modes all share a common structure, characterized by perturbations along the shock, together with streamwise-elongated regions of low and high momentum that originate at the shock foot and extend into the downstream flow. A linear superposition of these modes, with dynamics governed by their corresponding DMD eigenvalues, accurately captures the unsteadiness of the shock. In addition, DMD analysis shows that the downstream regions of low and high momentum are unsteady and that their unsteadiness is linked to the unsteadiness of the shock. The observed flow structures in the downstream flow are reminiscent of Gortler-like vortices that are present in this type of flow due to an underlying centrifugal instability, suggesting a possible physical mechanism for the low-frequency unsteadiness in shock wave/turbulent boundary layer interactions.

/pdf/low-frequency-dynamics-in-a-shock-induced-separated-flow-3pq659movy.pdf

Jonathan H. Tu

Papers

On dynamic mode decomposition: Theory and applications

On Dynamic Mode Decomposition: Theory and Applications

Variants of Dynamic Mode Decomposition: Boundary Condition, Koopman, and Fourier Analyses

Compressed sensing and dynamic mode decomposition

Low-frequency dynamics in a shock-induced separated flow