This is a tutorial on some basic non-asymptotic methods and concepts in random matrix theory. The reader will learn several tools for the analysis of the extreme singular values of random matrices with independent rows or columns. Many of these methods sprung off from the development of geometric functional analysis since the 1970's. They have applications in several fields, most notably in theoretical computer science, statistics and signal processing. A few basic applications are covered in this text, particularly for the problem of estimating covariance matrices in statistics and for validating probabilistic constructions of measurement matrices in compressed sensing. These notes are written particularly for graduate students and beginning researchers in different areas, including functional analysts, probabilists, theoretical statisticians, electrical engineers, and theoretical computer scientists.

Introduction to the non-asymptotic analysis of random matrices.

Concentration functions and inequalities Isoperimetric and functional examples Concentration and geometry Concentration in product spaces Entropy and concentration Transportation cost inequalities Sharp bounds of Gaussian and empirical processes Selected applications References Index

/pdf/the-concentration-of-measure-phenomenon-27qyoor2ef.pdf

The concentration of measure phenomenon

This paper presents new probability inequalities for sums of independent, random, self-adjoint matrices. These results place simple and easily verifiable hypotheses on the summands, and they deliver strong conclusions about the large-deviation behavior of the maximum eigenvalue of the sum. Tail bounds for the norm of a sum of random rectangular matrices follow as an immediate corollary. The proof techniques also yield some information about matrix-valued martingales.

In other words, this paper provides noncommutative generalizations of the classical bounds associated with the names Azuma, Bennett, Bernstein, Chernoff, Hoeffding, and McDiarmid. The matrix inequalities promise the same diversity of application, ease of use, and strength of conclusion that have made the scalar inequalities so valuable.

/pdf/user-friendly-tail-bounds-for-sums-of-random-matrices-unyakp5mw0.pdf

User-Friendly Tail Bounds for Sums of Random Matrices

High-dimensional probability offers insight into the behavior of random vectors, random matrices, random subspaces, and objects used to quantify uncertainty in high dimensions Drawing on ideas from probability, analysis, and geometry, it lends itself to applications in mathematics, statistics, theoretical computer science, signal processing, optimization, and more It is the first to integrate theory, key tools, and modern applications of high-dimensional probability Concentration inequalities form the core, and it covers both classical results such as Hoeffding's and Chernoff's inequalities and modern developments such as the matrix Bernstein's inequality It then introduces the powerful methods based on stochastic processes, including such tools as Slepian's, Sudakov's, and Dudley's inequalities, as well as generic chaining and bounds based on VC dimension A broad range of illustrations is embedded throughout, including classical and modern results for covariance estimation, clustering, networks, semidefinite programming, coding, dimension reduction, matrix completion, machine learning, compressed sensing, and sparse regression

High-Dimensional Probability: An Introduction with Applications in Data Science

Exponential and Information Inequalities- Gaussian Processes- Gaussian Model Selection- Concentration Inequalities- Maximal Inequalities- Density Estimation via Model Selection- Statistical Learning

Concentration Inequalities and Model Selection

We show that for any random matrix (X ij ) with independent mean zero entries E∥(X ij )∥ ≤ C(max √ΣEX 2 ij + max √ΣEX 2 ij + 4√ΣEX 4 ij ), where C is some universal constant.

https://www.ams.org/proc/2005-133-05/S0002-9939-04-07800-1/S0002-9939-04-07800-1.pdf

Some estimates of norms of random matrices

A Bernstein-type exponential inequality for (generalized) canonical U-statistics of order 2 is obtained, and the Rosenthal and Hoffmann-Jorgensen inequalities for sums of independent random variables are extended to (generalized) U-statistics of any order whose kernels are either nonnegative or canonical.

Exponential and Moment Inequalities for U-Statistics

Let a a ∈ [0, 1] and r ∈ [1, 2] satisfy relation r = 2/(2 − a). Let μ(dx)=c r n exp(-(|x1| r +|x2| r +...+|x n | r ))dx1dx2...dx n be a probability measure on the Euclidean space (R n , ‖ · ‖). We prove that there exists a universal constant C such that for any smooth real function f on R n and any p ∈ [1,2) 

$$E_\mu f^2 - (E_\mu \left| f \right|^p )^{2/p} \leqslant C(2 - p)^a E_\mu \left\| {\nabla f} \right\|^2$$

. We prove also that if for some probabilistic measure μ on R n the above inequality is satisfied for any p ∈ [1, 2) and any smooth f then for any h : R n → R such that |h(x)-h(y)|≤∥x-y∥ there is E μ |h| < ∞ and 

$$\mu (h - E_\mu h > \sqrt C \cdot t) \leqslant e^{ - Kt^r }$$

for t > 1, where K > 0 is some universal constant.

/pdf/between-sobolev-and-poincare-40jvrnsyzg.pdf

Between Sobolev and Poincaré

For the sum $S = \sum X_i$ of a sequence $(X_i)$ of independent symmetric (or nonnegative) random variables, we give lower and upper estimates of moments of $S$. The estimates are exact, up to some universal constants, and extend the previous results for particular types of variables $X_i$.

/pdf/estimation-of-moments-of-sums-of-independent-real-random-25auapvlcn.pdf

Estimation of moments of sums of independent real random variables

We derive two-sided estimates on moments and tails of Gaussian chaoses, that is, random variables of the form ∑ai1,…,idgi1⋯gid, where gi are i.i.d. ${\mathcal{N}}(0,1)$ r.v.’s. Estimates are exact up to constants depending on d only.

https://projecteuclid.org/download/pdfview_1/euclid.aop/1171377444

Rafał Latała

Papers

Some estimates of norms of random matrices

Exponential and Moment Inequalities for U-Statistics

Between Sobolev and Poincaré

Estimation of moments of sums of independent real random variables

Estimates of moments and tails of Gaussian chaoses