Probability Distributions.- Linear Models for Regression.- Linear Models for Classification.- Neural Networks.- Kernel Methods.- Sparse Kernel Machines.- Graphical Models.- Mixture Models and EM.- Approximate Inference.- Sampling Methods.- Continuous Latent Variables.- Sequential Data.- Combining Models.

Pattern Recognition and Machine Learning

/pdf/convex-analysisnoer-san-nojin-zhan-nituite-5dl2rbmoyr.pdf

Convex Analysisの二,三の進展について

Couplings and changes of variables.- Three examples of coupling techniques.- The founding fathers of optimal transport.- Qualitative description of optimal transport.- Basic properties.- Cyclical monotonicity and Kantorovich duality.- The Wasserstein distances.- Displacement interpolation.- The Monge-Mather shortening principle.- Solution of the Monge problem I: global approach.- Solution of the Monge problem II: Local approach.- The Jacobian equation.- Smoothness.- Qualitative picture.- Optimal transport and Riemannian geometry.- Ricci curvature.- Otto calculus.- Displacement convexity I.- Displacement convexity II.- Volume control.- Density control and local regularity.- Infinitesimal displacement convexity.- Isoperimetric-type inequalities.- Concentration inequalities.- Gradient flows I.- Gradient flows II: Qualitative properties.- Gradient flows III: Functional inequalities.- Synthetic treatment of Ricci curvature.- Analytic and synthetic points of view.- Convergence of metric-measure spaces.- Stability of optimal transport.- Weak Ricci curvature bounds I: Definition and Stability.- Weak Ricci curvature bounds II: Geometric and analytic properties.

Optimal Transport: Old and New

The author's preface gives an outline: "This book is about weakconvergence methods in metric spaces, with applications sufficient to show their power and utility. The Introduction motivates the definitions and indicates how the theory will yield solutions to problems arising outside it. Chapter 1 sets out the basic general theorems, which are then specialized in Chapter 2 to the space C[0, l ] of continuous functions on the unit interval and in Chapter 3 to the space D [0, 1 ] of functions with discontinuities of the first kind. The results of the first three chapters are used in Chapter 4 to derive a variety of limit theorems for dependent sequences of random variables. " The book develops and expands on Donsker's 1951 and 1952 papers on the invariance principle and empirical distributions. The basic random variables remain real-valued although, of course, measures on C[0, l ] and D[0, l ] are vitally used. Within this framework, there are various possibilities for a different and apparently better treatment of the material. More of the general theory of weak convergence of probabilities on separable metric spaces would be useful. Metrizability of the convergence is not brought up until late in the Appendix. The close relation of the Prokhorov metric and a metric for convergence in probability is (hence) not mentioned (see V. Strassen, Ann. Math. Statist. 36 (1965), 423-439; the reviewer, ibid. 39 (1968), 1563-1572). This relation would illuminate and organize such results as Theorems 4.1, 4.2 and 4.4 which give isolated, ad hoc connections between weak convergence of measures and nearness in probability. In the middle of p. 16, it should be noted that C*(S) consists of signed measures which need only be finitely additive if 5 is not compact. On p. 239, where the author twice speaks of separable subsets having nonmeasurable cardinal, he means "discrete" rather than "separable." Theorem 1.4 is Ulam's theorem that a Borel probability on a complete separable metric space is tight. Theorem 1 of Appendix 3 weakens completeness to topological completeness. After mentioning that probabilities on the rationals are tight, the author says it is an

Convergence of probability measures

This purpose of this introductory paper is threefold. First, it introduces the Monte Carlo method with emphasis on probabilistic machine learning. Second, it reviews the main building blocks of modern Markov chain Monte Carlo simulation, thereby providing and introduction to the remaining papers of this special issue. Lastly, it discusses new interesting research horizons.

/pdf/an-introduction-to-mcmc-for-machine-learning-22slt5b6x5.pdf

An introduction to MCMC for machine learning

The geometry and analysis that is discussed in this book extends to classical results for general discrete or Lie groups, and the methods used are analytical but have little to do with what is described these days as real analysis. Most of the results described in this book have a dual formulation; they have a 'discrete version' related to a finitely generated discrete group, and a continuous version related to a Lie group. The authors chose to centre this book around Lie groups but could quite easily have pushed it in several other directions as it interacts with opetators, and probability theory, as well as with group theory. This book will serve as an excellent basis for graduate courses in Lie groups, Markov chains or potential theory.

Analysis and Geometry on Groups

This is an expository paper on the use of logarithmic Sobolev inequalities for bounding rates of convergence of Markov chains on finite state spaces to their stationary distributions. Logarithmic Sobolev inequalities complement eigenvalue techniques and work for nonreversible chains in continuous time. Some aspects of the theory simplify considerably with finite state spaces and we are able to give a self-contained development. Examples of applications include the study of a Metropolis chain for the binomial distribution, sharp results for natural chains on the box of side n in d dimensions and improved rates for exclusion processes. We also show that for most r-regular graphs the log-Sobolev constant is of smaller order than the spectral gap. The log-Sobolev constant of the asymmetric two-point space is computed exactly as well as the log-Sobolev constant of the complete graph on n points.

/pdf/logarithmic-sobolev-inequalities-for-finite-markov-chains-3ax1tqsvza.pdf

Logarithmic sobolev inequalities for finite markov chains

Preface Introduction 1. Sobolev inequalities in Rn 2. Moser's elliptic Harnack Inequality 3. Sobolev inequalities on manifolds 4. Two applications 5. Parabolic Harnack inequalities.

/pdf/aspects-of-sobolev-type-inequalities-2ah3jgzi1m.pdf

Aspects of Sobolev-type inequalities

A note on Poincaré, Sobolev, and Harnack inequalities

By symmetry, P has eigenvalues 1 = I03 > I381 > ?> I 31xI- 1 2 -1. This paper develops methods for getting upper and lower bounds on 8i3 by comparison with a second reversible chain on the same state space. This extends the ideas introduced in Diaconis and Saloff-Coste (1993), where random walks on finite groups were considered. The bounds involve geometric properties such as the diameter and covering number of an associated graph along the lines of Diaconis and Stroock (1991). The main application gives a sharp upper bound on the second eigenvalue of the symmetric exclusion process. Thus, let S0 be a connected undirected graph with n vertices. For simplicity, we assume in this introduction that SW is regular. To start, r unlabelled particles are placed in an initial configuration, 1 < r < n. At each step, a particle is chosen at random; then one of the neighboring sites of this particle is chosen at random. If the neighboring site is unoccupied, the chosen particle is moved there; if the neighboring site is occupied, the system stays as it was. This is a reversible Markov chain on the r-sets of {1, 2, . . ., n} with uniform stationary distribution. Liggett (1985) gives background and motivation (he focuses on infinite systems). Fill (1991) gives bounds on the second eigenvalue of the labeled exclusion process on the finite circle ZZn 1 We study this chain by comparison with a second Markov chain on r-sets that proceeds by picking a particle at random, picking an unoccupied site at random (not necessarily a neighboring site) and moving the particle to the unoccupied site. This is a well studied chain (the Bernoulli-Laplace model for diffusion). Its eigenvalues are known. We show that the comparison techniques apply to give upper bounds on the eigenvalues of the exclusion

/pdf/comparison-theorems-for-reversible-markov-chains-4pc4m0g5sa.pdf

Laurent Saloff-Coste

Papers

Analysis and Geometry on Groups

Logarithmic sobolev inequalities for finite markov chains

Aspects of Sobolev-type inequalities

A note on Poincaré, Sobolev, and Harnack inequalities

Comparison theorems for reversible markov chains