Current methods for inferring population structure from genetic data do not provide formal significance tests for population differentiation. We discuss an approach to studying population structure (principal components analysis) that was first applied to genetic data by Cavalli-Sforza and colleagues. We place the method on a solid statistical footing, using results from modern statistics to develop formal significance tests. We also uncover a general “phase change” phenomenon about the ability to detect structure in genetic data, which emerges from the statistical theory we use, and has an important implication for the ability to discover structure in genetic data: for a fixed but large dataset size, divergence between two populations (as measured, for example, by a statistic like FST) below a threshold is essentially undetectable, but a little above threshold, detection will be easy. This means that we can predict the dataset size needed to detect structure.

/pdf/population-structure-and-eigenanalysis-46f957xkuu.pdf

Population structure and eigenanalysis

Before the 1960s, all anti-Stokes emissions, which were known to exist, involved emission energies in excess of excitation energies by only a few kT. They were linked to thermal population of energy states above excitation states by such an energy amount. It was the well-known case of anti-Stokes emission for the so-called thermal bands or in the Raman effect for the well-known anti-Stokes sidebands. Thermoluminescence, where traps are emptied by excitation energies of the order of kT, also constituted a field of anti-Stokes emission of its own. Superexcitation, i.e., raising an already excited electron to an even higher level by excited-state absorption (ESA), was also known but with very weak emissions. These types of well-known anti-Stokes processes have been reviewed in classical textbooks on luminescence.1 All fluorescence light emitters usually follow the well-known principle of the Stokes law which simply states that excitation photons are at a higher energy than emitted ones or, in other words, that output photon energy is weaker than input photon energy. This, in a sense, is an indirect statement that efficiency cannot be larger than 1. This principle is

Upconversion and Anti-Stokes Processes with f and d Ions in Solids

Recent interest in aspects common to quantum information and condensed matter has prompted a flurry of activity at the border of these disciplines that were far distant until a few years ago. Numerous interesting questions have been addressed so far. Here an important part of this field, the properties of the entanglement in many-body systems, are reviewed. The zero and finite temperature properties of entanglement in interacting spin, fermion, and boson model systems are discussed. Both bipartite and multipartite entanglement will be considered. In equilibrium entanglement is shown tightly connected to the characteristics of the phase diagram. The behavior of entanglement can be related, via certain witnesses, to thermodynamic quantities thus offering interesting possibilities for an experimental test. Out of equilibrium entangled states are generated and manipulated by means of many-body Hamiltonians.

/pdf/entanglement-in-many-body-systems-sqpatuxt14.pdf

Entanglement in many-body systems

This is a tutorial on some basic non-asymptotic methods and concepts in random matrix theory. The reader will learn several tools for the analysis of the extreme singular values of random matrices with independent rows or columns. Many of these methods sprung off from the development of geometric functional analysis since the 1970's. They have applications in several fields, most notably in theoretical computer science, statistics and signal processing. A few basic applications are covered in this text, particularly for the problem of estimating covariance matrices in statistics and for validating probabilistic constructions of measurement matrices in compressed sensing. These notes are written particularly for graduate students and beginning researchers in different areas, including functional analysts, probabilists, theoretical statisticians, electrical engineers, and theoretical computer scientists.

Introduction to the non-asymptotic analysis of random matrices.

Let x(1) denote the square of the largest singular value of an n × p matrix X, all of whose entries are independent standard Gaussian variates. Equivalently, x(1) is the largest principal component variance of the covariance matrix $X'X$, or the largest eigenvalue of a p­variate Wishart distribution on n degrees of freedom with identity covariance. Consider the limit of large p and n with $n/p = \gamma \ge 1$. When centered by $\mu_p = (\sqrt{n-1} + \sqrt{p})^2$ and scaled by $\sigma_p = (\sqrt{n-1} + \sqrt{p})(1/\sqrt{n-1} + 1/\sqrt{p}^{1/3}$, the distribution of x(1) approaches the Tracey-Widom law of order 1, which is defined in terms of the Painleve II differential equation and can be numerically evaluated and tabulated in software. Simulations show the approximation to be informative for n and p as small as 5. The limit is derived via a corresponding result for complex Wishart matrices using methods from random matrix theory. The result suggests that some aspects of large p multivariate distribution theory may be easier to apply in practice than their fixed p counterparts.

/pdf/on-the-distribution-of-the-largest-eigenvalue-in-principal-3i23kzfus6.pdf

On the distribution of the largest eigenvalue in principal components analysis

Scaling level-spacing distribution functions in the “bulk of the spectrum” in random matrix models ofN×N hermitian matrices and then going to the limitN→∞ leads to the Fredholm determinant of thesine kernel sinπ(x−y)/π(x−y). Similarly a scaling limit at the “edge of the spectrum” leads to theAiry kernel [Ai(x)Ai(y)−Ai′(x)Ai(y)]/(x−y). In this paper we derive analogues for this Airy kernel of the following properties of the sine kernel: the completely integrable system of P.D.E.'s found by Jimbo, Miwa, Mori, and Sato; the expression, in the case of a single interval, of the Fredholm determinant in terms of a Painleve transcendent; the existence of a commuting differential operator; and the fact that this operator can be used in the derivation of asymptotics, for generaln, of the probability that an interval contains preciselyn eigenvalues.

/pdf/level-spacing-distributions-and-the-airy-kernel-2g64c94n7y.pdf

Level spacing distributions and the Airy kernel

The focus of this paper is on the probability,Eβ(O;J), that a setJ consisting of a finite union of intervals contains no eigenvalues for the finiteN Gaussian Orthogonal (β=1) and Gaussian Symplectic (β=4) Ensembles and their respective scaling limits both in the bulk and at the edge of the spectrum. We show how these probabilities can be expressed in terms of quantities arising in the corresponding unitary (β=2) ensembles. Our most explicit new results concern the distribution of the largest eigenvalue in each of these ensembles. In the edge scaling limit we show that these largest eigenvalue distributions are given in terms of a particular Painleve II function.

/pdf/on-orthogonal-and-symplectic-matrix-ensembles-30l4x37qc2.pdf

On orthogonal and symplectic matrix ensembles

Here I = S j (a2j 1,a2j) andI(y) is the characteristic function of the set I. In the Gaussian Unitary Ensemble (GUE) the probability that no eigenvalues lie in I is equal to �(a). Also �(a) is a tau-function and we present a new simplified derivation of the system of nonlinear completely integrable equations (the aj's are the independent variables) that were first derived by Jimbo, Miwa, Mori, and Sato in 1980. In the case of a single interval these equations are reducible to a Painleve V equation. For large s we give an asymptotic formula for E2(n;s), which is the probability in the GUE that exactly n eigenvalues lie in an interval of length s. These notes provide an introduction to that aspect of the theory of random matrices dealing with the distribution of eigenvalues. To first orient the reader, we present in Sec. II some numerical experiments that illustrate some of the basic aspects of the subject. In Sec. III we introduce the invariant measures for the three "circular ensembles" involving unitary matrices. We also define the level spacing distributions and express these distributions in terms of a particular Fredholm determinant. In Sec. IV we explain how these measures are modified for the orthogonal polynomial ensembles. In Sec. V we discuss the universality of these level spacing distribution functions in a particular scaling limit. The discussion up to this point (with the possible exception of Sec. V) follows the well-known path pioneered by Hua, Wigner, Dyson, Mehta and others who first developed this theory (see, e.g., the reprint volume of Porter (34) and Hua (17)). This, and much more, is discussed in Mehta's book (25)—the classic reference in the subject. An important development in random matrices was the discovery by Jimbo, Miwa, Mori, and Sato (21) (hereafter referred to as JMMS) that the basic Fredholm determinant mentioned above is a �-function in the sense of the Kyoto School. Though it has been some twelve years since (21) was published, these results are not widely appreciated by the practitioners of random matrices. This is due no doubt to the complexity of their paper. The methods of JMMS are methods of discovery; but now that we know the result, simpler proofs can be constructed. In Sec. VI we give such a proof of the JMMS equations. Our proof is a simplification and generalization of Mehta's (27) simplified proof of the single interval case. Also our methods build on the earlier work of Its, Izergin, Korepin, and Slavnov (18) and Dyson (12). We include in this section a discussion of the connection between the JMMS equations and the integrable Hamiltonian systems that appear in the geometry of quadrics and spectral theory as developed by Moser (31). This section concludes with a discussion of the case of a single interval (viz., probability that exactly n eigenvalues lie in a given interval). In this case the JMMS equations can be reduced to a single ordinary differential equation—the Painleve V equation. Finally, in Sec. VII we discuss the asymptotics in the case of a large single interval of the various level spacing distribution functions (4,38,28). In this analysis both the Painleve representation and new results in Toeplitz/Wiener- Hopf theory are needed to produce these asymptotics. We also give an approach based on the asymptotics of the eigenvalues of the basic linear integral operator (14,25,35). These results are then compared with the continuum model calculations of Dyson (12).

/pdf/introduction-to-random-matrices-25zedrcbop.pdf

Introduction to Random Matrices

We compute exactly the spin-spin correlation functions 〈σ0,0σM,N〉 for the two-dimensional Ising model on a square lattice in zero magnetic field for T>Tc and T<Tc. We then analyze the correlation functions in the scaling limit T→Tc,M2+N2→∞ such that (T−Tc) is fixed. In this scaling limit 〈σ0,0σM,N〉=R−14F±(t)+R−54F1±(t)+o(R−54), where t is the scaling variable Rξ and F±(t) and F1±(t) are the scaling functions (ξ is the correlation length). We derive exact expressions for these scaling functions, in terms of a Painleve function of the third kind and analyze both the small- and large-t behavior. A table of values for F±(t) (good to ten significant digits) is also given. As an application we computer the coefficients C0± and C1± in the expansion kBTχ(T)=C0±|1−TcT|−74+C1±|1−TcT|−34+O(1) of the zero-field susceptibility χ(T) as T→Tc±.

Spin spin correlation functions for the two-dimensional Ising model: Exact theory in the scaling region

Scaling models of randomN×N hermitian matrices and passing to the limitN→∞ leads to integral operators whose Fredholm determinants describe the statistics of the spacing of the eigenvalues of hermitian matrices of large order. For the Gaussian Unitary Ensemble, and for many others'as well, the kernel one obtains by scaling in the “bulk” of the spectrum is the “sine kernel”$\frac{{\sin \pi (x - y)}}{{\pi (x - y)}}$. Rescaling the GUE at the “edge” of the spectrum leads to the kernel$\frac{{Ai(x)Ai'(y) - Ai'(x)Ai(y)}}{{x - y}}$, where Ai is the Airy function. In previous work we found several analogies between properties of this “Airy kernel” and known properties of the sine kernel: a system of partial differential equations associated with the logarithmic differential of the Fredholm determinant when the underlying domain is a union of intervals; a representation of the Fredholm determinant in terms of a Painleve transcendent in the case of a single interval; and, also in this case, asymptotic expansions for these determinants and related quantities, achieved with the help of a differential operator which commutes with the integral operator. In this paper we show that there are completely analogous properties for a class of kernels which arise when one rescales the Laguerre or Jacobi ensembles at the edge of the spectrum, namely

$$\frac{{J_\alpha (\sqrt x )\sqrt y J'_\alpha (\sqrt y ) - \sqrt x J'_\alpha (\sqrt x )J_\alpha (\sqrt y )}}{{2(x - y)}},$$

, whereJα(z) is the Bessel function of order α. In the cases α=∓1/2 these become, after a variable change, the kernels which arise when taking scaling limits in the bulk of the spectrum for the Gaussian orthogonal and symplectic ensembles. In particular, an asymptotic expansion we derive will generalize ones found by Dyson for the Fredholm determinants of these kernels.

/pdf/level-spacing-distributions-and-the-bessel-kernel-3mjnafxjs2.pdf

Craig A. Tracy

Papers

Level spacing distributions and the Airy kernel

On orthogonal and symplectic matrix ensembles

Introduction to Random Matrices

Spin spin correlation functions for the two-dimensional Ising model: Exact theory in the scaling region

Level spacing distributions and the Bessel kernel