Large deviations conditioned on large deviations I: Markov chain and Langevin equation
TL;DR: In this paper, the authors present a systematic analysis of stochastic processes conditioned on an empirical measure defined in a time interval for large values of T. The authors build their analysis starting from a discrete time Markov chain and show how conditioning on a value of T modifies the dynamics.
Abstract: We present a systematic analysis of stochastic processes conditioned on an empirical measure $Q_T$ defined in a time interval $[0,T]$ for large $T$. We build our analysis starting from a discrete time Markov chain. Results for a continuous time Markov process and Langevin dynamics are derived as limiting cases. We show how conditioning on a value of $Q_T$ modifies the dynamics. For a Langevin dynamics with weak noise, we introduce conditioned large deviations functions and calculate them using either a WKB method or a variational formulation. This allows us, in particular, to calculate the typical trajectory and the fluctuations around this optimal trajectory when conditioned on a certain value of $Q_T$.
Citations
More filters
TL;DR: Van Kampen as mentioned in this paper provides an extensive graduate-level introduction which is clear, cautious, interesting and readable, and could be expected to become an essential part of the library of every physical scientist concerned with problems involving fluctuations and stochastic processes.
Abstract: N G van Kampen 1981 Amsterdam: North-Holland xiv + 419 pp price Dfl 180 This is a book which, at a lower price, could be expected to become an essential part of the library of every physical scientist concerned with problems involving fluctuations and stochastic processes, as well as those who just enjoy a beautifully written book. It provides an extensive graduate-level introduction which is clear, cautious, interesting and readable.
3,647 citations
TL;DR: In this paper, the authors focus on rare trajectories presenting an atypical value of the observable, that they study through a biased dynamics in a large-deviation framework and determine explicitly the effective probability-conserving dynamics which makes rare trajectory of the original dynamics become typical trajectories of the effective one.
Abstract: In this work we focus on fluctuations of time-integrated observables for a particle diffusing in a one-dimensional periodic potential in the weak-noise asymptotics. Our interest goes to rare trajectories presenting an atypical value of the observable, that we study through a biased dynamics in a large-deviation framework. We determine explicitly the effective probability-conserving dynamics which makes rare trajectories of the original dynamics become typical trajectories of the effective one. Our approach makes use of a weak-noise path-integral description in which the action is minimised by the rare trajectories of interest. For `current-type' additive observables, we find the emergence of a propagative trajectory minimising the action for large enough deviations, revealing the existence of a dynamical phase transition at a fluctuating level. In addition, we provide a new method to determine the scaled cumulant generating function of the observable without having to optimise the action. It allows one to show that the weak-noise and the large-time limits commute in this problem. Finally, we show how the biased dynamics can be mapped in practice to an effective driven dynamics, which takes the form of a driven Langevin dynamics in an effective potential. The non-trivial shape of this effective potential is key to understand the link between the dynamical phase transition in the large deviations of current and the standard depinning transition of a particle in a tilted potential.
55 citations
TL;DR: A general approach to adaptively construct a dynamics that efficiently samples atypical events is presented, exploiting the methods of reinforcement learning (RL), which refers to the set of machine learning techniques aimed at finding the optimal behaviour to maximise a reward associated with the dynamics.
Abstract: Very often when studying non-equilibrium systems one is interested in analysing dynamical behaviour that occurs with very low probability, so called rare events. In practice, since rare events are by definition atypical, they are often difficult to access in a statistically significant way. What are required are strategies to "make rare events typical" so that they can be generated on demand. Here we present such a general approach to adaptively construct a dynamics that efficiently samples atypical events. We do so by exploiting the methods of reinforcement learning (RL), which refers to the set of machine learning techniques aimed at finding the optimal behaviour to maximise a reward associated with the dynamics. We consider the general perspective of dynamical trajectory ensembles, whereby rare events are described in terms of ensemble reweighting. By minimising the distance between a reweighted ensemble and that of a suitably parametrised controlled dynamics we arrive at a set of methods similar to those of RL to numerically approximate the optimal dynamics that realises the rare behaviour of interest. As simple illustrations we consider in detail the problem of excursions of a random walker, for the case of rare events with a finite time horizon; and the problem of a studying current statistics of a particle hopping in a ring geometry, for the case of an infinite time horizon. We discuss natural extensions of the ideas presented here, including to continuous-time Markov systems, first passage time problems and non-Markovian dynamics.
37 citations
Cites background from "Large deviations conditioned on lar..."
...This cound by done by considering conditioned ensembles for each of its possible values, however, this is often a difficult task even for a single value [34, 35]....
[...]
TL;DR: In this paper, the authors studied the statistical properties of the long-time dynamics of the rule 54 reversible cellular automata (CA), driven stochastically at its boundaries, and showed that all instances of boundary driving the CA dynamics occur at the point of phase coexistence between competing active and inactive dynamical phases, similar to what happens in more standard KCMs.
Abstract: We study the statistical properties of the long-time dynamics of the rule 54 reversible cellular automaton (CA), driven stochastically at its boundaries. This CA can be considered as a discrete-time and deterministic version of the Fredrickson-Andersen kinetically constrained model (KCM). By means of a matrix product ansatz, we compute the exact large deviation cumulant generating functions for a wide range of time-extensive observables of the dynamics, together with their associated rate functions and conditioned long-time distributions over configurations. We show that for all instances of boundary driving the CA dynamics occurs at the point of phase coexistence between competing active and inactive dynamical phases, similar to what happens in more standard KCMs. We also find the exact finite size scaling behavior of these trajectory transitions, and provide the explicit "Doob-transformed" dynamics that optimally realizes rare dynamical events.
28 citations
TL;DR: In this article, a path integral formalism over a Lagrangian functional of concentrations and chemical fluxes is proposed, whose trajectories correspond to the most likely evolution of the system given its boundary conditions.
Abstract: Chemical reaction networks offer a natural nonlinear generalisation of linear Markov jump processes on a finite state-space. In this paper, we analyse the dynamical large deviations of such models, starting from their microscopic version, the chemical master equation. By taking a large-volume limit, we show that those systems can be described by a path integral formalism over a Lagrangian functional of concentrations and chemical fluxes. This Lagrangian is dual to a Hamiltonian, whose trajectories correspond to the most likely evolution of the system given its boundary conditions. The same can be done for a system biased on time-averaged concentrations and currents, yielding a biased Hamiltonian whose trajectories are optimal paths conditioned on those observables. The appropriate boundary conditions turn out to be mixed, so that, in the long time limit, those trajectories converge to well-defined attractors. We are then able to identify the largest value that the Hamiltonian takes over those attractors with the scaled cumulant generating function of our observables, providing a non-linear equivalent to the well-known Donsker-Varadhan formula for jump processes. On that basis, we prove that chemical reaction networks that are deterministically multistable generically undergo first-order dynamical phase transitions in the vicinity of zero bias. We illustrate that fact through a simple bistable model called the Schl\"ogl model, as well as multistable and unstable generalisations of it, and we make a few surprising observations regarding the stability of deterministic fixed points, and the breaking of ergodicity in the large-volume limit.
24 citations
References
More filters
01 Jan 1992
Abstract: Preface to the first edition. Preface to the second edition. Abbreviated references. I. Stochastic variables. II. Random events. III. Stochastic processes. IV. Markov processes. V. The master equation. VI. One-step processes. VII. Chemical reactions. VIII. The Fokker-Planck equation. IX. The Langevin approach. X. The expansion of the master equation. XI. The diffusion type. XII. First-passage problems. XIII. Unstable systems. XIV. Fluctuations in continuous systems. XV. The statistics of jump events. XVI. Stochastic differential equations. XVII. Stochastic behavior of quantum systems.
6,887 citations
TL;DR: Van Kampen as mentioned in this paper provides an extensive graduate-level introduction which is clear, cautious, interesting and readable, and could be expected to become an essential part of the library of every physical scientist concerned with problems involving fluctuations and stochastic processes.
Abstract: N G van Kampen 1981 Amsterdam: North-Holland xiv + 419 pp price Dfl 180 This is a book which, at a lower price, could be expected to become an essential part of the library of every physical scientist concerned with problems involving fluctuations and stochastic processes, as well as those who just enjoy a beautifully written book. It provides an extensive graduate-level introduction which is clear, cautious, interesting and readable.
3,647 citations
"Large deviations conditioned on lar..." refers background or methods or result in this paper
...One may alternatively derive the same results using the Kramers-Moyal expansion [50] of the continuous time Markov process in Appendix B....
[...]
...This is known as the Ornstein-Uhlenbeck process [50]....
[...]
...The Perron-Frobenius theorem [50] ensures that the largest eigenvalue of Mλ is positive and non-degenerate, and all components of the associated right and left eigenvectors are positive....
[...]
...It is well known [50] that the probability Pt(x) of the process Xt to be in x at time t follows a Fokker-Planck equation d dt Pt(x) = L0 · Pt(x) := − d dx [F (x)Pt(x)] + 2 d(2) dx2 Pt(x) (27)...
[...]
TL;DR: In this paper, the authors introduce the concept of large deviations for random variables with a finite state space, which is a generalization of the notion of large deviation for random vectors.
Abstract: I: Large Deviations and Statistical Mechanics.- I. Introduction to Large Deviations.- I.1. Overview.- I.2. Large Deviations for I.I.D. Random Variables with a Finite State Space.- I.3. Levels-1 and 2 for Coin Tossing.- I.4. Levels-1 and 2 for I.I.D. Random Variables with a Finite State Space.- I.S. Level-3: Empirical Pair Measure.- I.6. Level-3: Empirical Process.- I.7. Notes.- I.B. Problems.- II. Large Deviation Property and Asymptotics of Integrals.- II.1. Introduction.- II.2. Levels-1, 2, and 3 Large Deviations for I.I.D. Random Vectors.- II.3. The Definition of Large Deviation Property.- II.4. Statement of Large Deviation Properties for Levels-1, 2, and 3.- II.5. Contraction Principles.- II.6. Large Deviation Property for Random Vectors and Exponential Convergence.- II.7. Varadhan's Theorem on the Asymptotics of Integrals.- II.8. Notes.- II.9. Problems.- III. Large Deviations and the Discrete Ideal Gas.- III.1. Introduction.- III.2. Physics Prelude: Thermodynamics.- III.3. The Discrete Ideal Gas and the Microcanonical Ensemble.- III.4. Thermodynamic Limit, Exponential Convergence, and Equilibrium Values.- III.5. The Maxwell-Boltzmann Distribution and Temperature.- III.6. The Canonical Ensemble and Its Equivalence with the Microcanonical Ensemble.- III.7. A Derivation of a Thermodynamic Equation.- III.8. The Gibbs Variational Formula and Principle.- III.9. Notes.- III.10. Problems.- IV. Ferromagnetic Models on ?.- IV.1. Introduction.- IV.2. An Overview of Ferromagnetic Models.- IV.3. Finite-Volume Gibbs States on ?.- IV.4. Spontaneous Magnetization for the Curie-Weiss Model.- IV.5. Spontaneous Magnetization for General Ferromagnets on ?.- IV.6. Infinite-Volume Gibbs States and Phase Transitions.- IV.7. The Gibbs Variational Formula and Principle.- IV.8. Notes.- IV.9. Problems.- V. Magnetic Models on ?D and on the Circle.- V.1. Introduction.- V.2. Finite-Volume Gibbs States on ?D, D ? 1.- V.3. Moment Inequalities.- V.4. Properties of the Magnetization and the Gibbs Free Energy.- V.5. Spontaneous Magnetization on ?D, D ? 2, Via the Peierls Argument.- V.6. Infinite-Volume Gibbs States and Phase Transitions.- V.7. Infinite-Volume Gibbs States and the Central Limit Theorem.- V.8. Critical Phenomena and the Breakdown of the Central Limit Theorem.- V.9. Three Faces of the Curie-Weiss Model.- V.10. The Circle Model and Random Waves.- V.11. A Postscript on Magnetic Models.- V.12. Notes.- V.13. Problems.- II: Convexity and Proofs of Large Deviation Theorems.- VI. Convex Functions and the Legendre-Fenchel Transform.- VI.1. Introduction.- VI.2. Basic Definitions.- VI.3. Properties of Convex Functions.- VI.4. A One-Dimensional Example of the Legendre-Fenchel Transform.- VI.5. The Legendre-Fenchel Transform for Convex Functions on ?d.- VI.6. Notes.- VI.7. Problems.- VII. Large Deviations for Random Vectors.- VII.1. Statement of Results.- VII.2. Properties of IW.- VII.3. Proof of the Large Deviation Bounds for d = 1.- VII.4. Proof of the Large Deviation Bounds for d ? 1.- VII.5. Level-1 Large Deviations for I.I.D. Random Vectors.- VII.6. Exponential Convergence and Proof of Theorem II.6.3.- VII.7. Notes.- VII.8. Problems.- VIII. Level-2 Large Deviations for I.I.D. Random Vectors.- VIII.1. Introduction.- VIII.2. The Level-2 Large Deviation Theorem.- VIII.3. The Contraction Principle Relating Levels-1 and 2 (d = 1).- VIII.4. The Contraction Principle Relating Levels-1 and 2 (d ? 2).- VIII.5. Notes.- VIII.6. Problems.- IX. Level-3 Large Deviations for I.I.D. Random Vectors.- IX.1. Statement of Results.- IX.2. Properties of the Level-3 Entropy Function.- IX.3. Contraction Principles.- IX.4. Proof of the Level-3 Large Deviation Bounds.- IX.5. Notes.- IX.6. Problems.- Appendices.- Appendix A: Probability.- A.1. Introduction.- A.2. Measurability.- A.3. Product Spaces.- A.4. Probability Measures and Expectation.- A.S. Convergence of Random Vectors.- A.6. Conditional Expectation, Conditional Probability, and Regular Conditional Distribution.- A.7. The Kolmogorov Existence Theorem.- A.8. Weak Convergence of Probability Measures on a Metric Space.- Appendix B: Proofs of Two Theorems in Section II.7.- B.1. Proof of Theorem II.7.1.- B.2. Proof of Theorem II.7.2.- Appendix C: Equivalent Notions of Infinite-Volume Measures for Spin Systems.- C.1. Introduction.- C.2. Two-Body Interactions and Infinite-Volume Gibbs States.- C.3. Many-Body Interactions and Infinite-Volume Gibbs States.- C.4. DLR States.- C.5. The Gibbs Variational Formula and Principle.- C.6. Solution of the Gibbs Variational Formula for Finite-Range Interactions on ?.- Appendix D: Existence of the Specific Gibbs Free Energy.- D.1. Existence Along Hypercubes.- D.2. An Extension.- List of Frequently Used Symbols.- References.- Author Index.
1,626 citations