There is, I think, something ethereal about i —the square root of minus one. I remember first hearing about it at school. It seemed an odd beast at that time—an intruder hovering on the edge of reality.

Usually familiarity dulls this sense of the bizarre, but in the case of i it was the reverse: over the years the sense of its surreal nature intensified. It seemed that it was impossible to write mathematics that described the real world in …

I and i

N G van Kampen 1981 Amsterdam: North-Holland xiv + 419 pp price Dfl 180 This is a book which, at a lower price, could be expected to become an essential part of the library of every physical scientist concerned with problems involving fluctuations and stochastic processes, as well as those who just enjoy a beautifully written book. It provides an extensive graduate-level introduction which is clear, cautious, interesting and readable.

https://iopscience.iop.org/article/10.1088/0031-9112/34/4/040/pdf

Stochastic Processes in Physics and Chemistry

Extracting governing equations from data is a central challenge in many diverse areas of science and engineering. Data are abundant whereas models often remain elusive, as in climate science, neuroscience, ecology, finance, and epidemiology, to name only a few examples. In this work, we combine sparsity-promoting techniques and machine learning with nonlinear dynamical systems to discover governing equations from noisy measurement data. The only assumption about the structure of the model is that there are only a few important terms that govern the dynamics, so that the equations are sparse in the space of possible functions; this assumption holds for many physical systems in an appropriate basis. In particular, we use sparse regression to determine the fewest terms in the dynamic governing equations required to accurately represent the data. This results in parsimonious models that balance accuracy with model complexity to avoid overfitting. We demonstrate the algorithm on a wide range of problems, from simple canonical systems, including linear and nonlinear oscillators and the chaotic Lorenz system, to the fluid vortex shedding behind an obstacle. The fluid example illustrates the ability of this method to discover the underlying dynamics of a system that took experts in the community nearly 30 years to resolve. We also show that this method generalizes to parameterized systems and systems that are time-varying or have external forcing.

https://www.pnas.org/content/pnas/113/15/3932.full.pdf

Discovering governing equations from data by sparse identification of nonlinear dynamical systems

Summary. Markov chain Monte Carlo and sequential Monte Carlo methods have emerged as the two main tools to sample from high dimensional probability distributions. Although asymptotic convergence of Markov chain Monte Carlo algorithms is ensured under weak assumptions, the performance of these algorithms is unreliable when the proposal distributions that are used to explore the space are poorly chosen and/or if highly correlated variables are updated independently. We show here how it is possible to build efficient high dimensional proposal distributions by using sequential Monte Carlo methods. This allows us not only to improve over standard Markov chain Monte Carlo schemes but also to make Bayesian inference feasible for a large class of statistical models where this was not previously so. We demonstrate these algorithms on a non-linear state space model and a Levy-driven stochastic volatility model.

https://rss.onlinelibrary.wiley.com/doi/pdf/10.1111/j.1467-9868.2009.00736.x

Particle Markov chain Monte Carlo methods

We provide a framework for structural multiscale geometric organization of graphs and subsets of R(n). We use diffusion semigroups to generate multiscale geometries in order to organize and represent complex structures. We show that appropriately selected eigenfunctions or scaling functions of Markov matrices, which describe local transitions, lead to macroscopic descriptions at different scales. The process of iterating or diffusing the Markov matrix is seen as a generalization of some aspects of the Newtonian paradigm, in which local infinitesimal transitions of a system lead to global macroscopic descriptions by integration. We provide a unified view of ideas from data analysis, machine learning, and numerical analysis.

/pdf/geometric-diffusions-as-a-tool-for-harmonic-analysis-and-1144ex6tu5.pdf

Geometric diffusions as a tool for harmonic analysis and structure definition of data: Diffusion maps

Recent developments in multiscale computation allow the solution of ``coarse equations'' for the expected macroscopic behavior of microscopically/stochastically evolving particle distributions without ever obtaining these coarse equations in closed form. The closure is obtained ``on demand'' through appropriately initialized bursts of microscopic simulation. The effective coupling of microscopic simulators with macrosocopic behavior embodied in this approach requires certain decisions about the nature of the unavailable ``coarse equation''. Such decisions include (a) the determination of the highest spatial derivative active in the equation, (b) whether the coarse equation satisfies certain conservation laws, and (c) whether the coarse dynamics is Hamiltonian. These decisions affect the number and type of boundary conditions as well as the nature of the algorithms employed in good solution practice. In the absence of an explicit formula for the temporal derivative, we propose, implement and validate a simple scheme for deciding these and other similar questions about the coarse equation using only the microscopic simulator. Microscopic simulations under periodic boundary conditions are carried out for appropriately chosen families of random initial conditions; evaluating the sample variance of certain statistics over the simulation ensemble allows us to infer the highest order of spatial derivatives active in the coarse equation. In the same spirit we show how to determine whether a certain coarse conservation law exists or not, and we discuss plausibility tests for the existence of a coarse Hamiltonian or integrability. We argue that such schemes constitute an important part of the equation-free approach to multiscale computation.

Deciding the Nature of the "Coarse Equation" through Microscopic Simulations: the Baby-Bathwater Scheme

In recent years, individual-based/agent-based modeling has been applied to study a wide range of applications, ranging from engineering problems to phenomena in sociology, economics and biology. Simulating such agent-based models over extended spatiotemporal domains can be prohibitively expensive due to stochasticity and the presence of multiple scales. Nevertheless, many agent-based problems exhibit smooth behavior in space and time on a macroscopic scale, suggesting that a useful coarse-grained continuum model could be obtained. For such problems, the equation-free framework [16-18] can significantly reduce the computational cost. Patch dynamics is an essential component of this framework. This scheme is designed to perform numerical simulations of an unavailable macroscopic equation on macroscopic time and length scales; it uses appropriately initialized simulations of the fine-scale agent-based model in a number of small "patches", which cover only a fraction of the spatiotemporal domain. In this work, we construct a finite-volume-inspired conservative patch dynamics scheme and apply it to a financial market agent-based model based on the work of Omurtag and Sirovich [22]. We first apply our patch dynamics scheme to a continuum approximation of the agent-based model, to study its performance and analyze its accuracy. We then apply the scheme to the agent-based model itself. Our computational experiments indicate that here, typically, the patch dynamics-based simulation requires only 20% of the full agent-based simulation in space, and need occur over only 10% of the temporal domain.

On the acceleration of spatially distributed agent-based computations: a patch dynamics scheme

The adoption of detailed mechanisms for chemical kinetics often poses two types of severe challenges: First, the number of degrees of freedom is large; and second, the dynamics is characterized by widely disparate time scales. As a result, reactive flow solvers with detailed chemistry often become intractable even for large clusters of CPUs, especially when dealing with direct numerical simulation (DNS) of turbulent combustion problems. This has motivated the development of several techniques for reducing the complexity of such kinetics models, where eventually only a few variables are considered in the development of the simplified model. Unfortunately, no generally applicable a priori recipe for selecting suitable parameterizations of the reduced model is available, and the choice of slow variables often relies upon intuition and experience. We present an automated approach to this task, consisting of three main steps. First, the low dimensional manifold of slow motions is (approximately) sampled by brief simulations of the detailed model, starting from a rich enough ensemble of admissible initial conditions. Second, a global parameterization of the manifold is obtained through the Diffusion Map (DMAP) approach, which has recently emerged as a powerful tool in data analysis/machine learning. Finally, a simplified model is constructed and solved on the fly in terms of the above reduced (slow) variables. Clearly, closing this latter model requires nontrivial interpolation calculations, enabling restriction (mapping from the full ambient space to the reduced one) and lifting (mapping from the reduced space to the ambient one). This is a key step in our approach, and a variety of interpolation schemes are reported and compared. The scope of the proposed procedure is presented and discussed by means of an illustrative combustion example.

C. William Gear

Papers

Deciding the Nature of the "Coarse Equation" through Microscopic Simulations: the Baby-Bathwater Scheme

On the acceleration of spatially distributed agent-based computations: a patch dynamics scheme

Reduced models in chemical kinetics via nonlinear data-mining