Statistics for Spatial Data.

Convergence of Probability Measures. By P. Billingsley. Chichester, Sussex, Wiley, 1968. xii, 253 p. 9 1/4“. 117s.

Convergence of Probability Measures

Large datasets are increasingly common and are often difficult to interpret. Principal component analysis (PCA) is a technique for reducing the dimensionality of such datasets, increasing interpretability but at the same time minimizing information loss. It does so by creating new uncorrelated variables that successively maximize variance. Finding such new variables, the principal components, reduces to solving an eigenvalue/eigenvector problem, and the new variables are defined by the dataset at hand, not a priori , hence making PCA an adaptive data analysis technique. It is adaptive in another sense too, since variants of the technique have been developed that are tailored to various different data types and structures. This article will begin by introducing the basic ideas of PCA, discussing what it can and cannot do. It will then describe some variants of PCA and their application.

https://royalsocietypublishing.org/doi/pdf/10.1098/rsta.2015.0202

Principal component analysis: a review and recent developments

In this book, Andrew Harvey sets out to provide a unified and comprehensive theory of structural time series models. Unlike the traditional ARIMA models, structural time series models consist explicitly of unobserved components, such as trends and seasonals, which have a direct interpretation. As a result the model selection methodology associated with structural models is much closer to econometric methodology. The link with econometrics is made even closer by the natural way in which the models can be extended to include explanatory variables and to cope with multivariate time series. From the technical point of view, state space models and the Kalman filter play a key role in the statistical treatment of structural time series models. The book includes a detailed treatment of the Kalman filter. This technique was originally developed in control engineering, but is becoming increasingly important in fields such as economics and operations research. This book is concerned primarily with modelling economic and social time series, and with addressing the special problems which the treatment of such series poses. The properties of the models and the methodological techniques used to select them are illustrated with various applications. These range from the modellling of trends and cycles in US macroeconomic time series to to an evaluation of the effects of seat belt legislation in the UK.

Forecasting, Structural Time Series Models and the Kalman Filter

Regression models are frequently used to develop diagnostic, prognostic, and health resource utilization models in clinical, health services, outcomes, pharmacoeconomic, and epidemiologic research, and in a multitude of non-health-related areas. Regression models are also used to adjust for patient heterogeneity in randomized clinical trials, to obtain tests that are more powerful and valid than unadjusted treatment comparisons.

/pdf/regression-modeling-strategies-1jyn0f4r32.pdf

Regression Modeling Strategies

In regression problems where the number of predictors greatly exceeds the number of observations, conventional regression techniques may produce unsatisfactory results. We describe a technique called supervised principal components that can be applied to this type of problem. Supervised principal components is similar to conventional principal components analysis except that it uses a subset of the predictors selected based on their association with the outcome. Supervised principal components can be applied to regression and generalized regression problems, such as survival analysis. It compares favorably to other techniques for this type of problem, and can also account for the effects of other covariates and help identify which predictor variables are most important. We also provide asymptotic consistency results to help support our empirical findings. These methods could become important tools for DNA microarray data, where they may be used to more accurately diagnose and treat cancer.

/pdf/prediction-by-supervised-principal-components-2o3rdwea41.pdf

Prediction by supervised principal components

This paper deals with a multivariate Gaussian observation model where the eigenvalues of the covariance matrix are all one, except for a finite number which are larger. Of interest is the asymptotic behavior of the eigenvalues of the sample covariance matrix when the sample size and the dimension of the obser- vations both grow to infinity so that their ratio converges to a positive constant. When a population eigenvalue is above a certain threshold and of multiplicity one, the corresponding sample eigenvalue has a Gaussian limiting distribution. There is a "phase transition" of the sample eigenvectors in the same setting. Another contribution here is a study of the second order asymptotics of sample eigenvectors when corresponding eigenvalues are simple and sufficiently l arge.

/pdf/asymptotics-of-sample-eigenstructure-for-a-large-dimensional-18e6csue9h.pdf

Asymptotics of sample eigenstructure for a large dimensional spiked covariance model

This correspondence studies the statistical distribution of the signal-to-interference-plus-noise ratio (SINR) for the minimum mean-square error (MMSE) receiver in multiple-input multiple-output (MIMO) wireless communications. The channel model is assumed to be (transmit) correlated Rayleigh flat-fading with unequal powers. The SINR can be decomposed into two independent random variables: SINR=SINR/sup ZF/+T, where SINR/sup ZF/ corresponds to the SINR for a zero-forcing (ZF) receiver and has an exact Gamma distribution. This correspondence focuses on characterizing the statistical properties of T using the results from random matrix theory. First three asymptotic moments of T are derived for uncorrelated channels and channels with equicorrelations. For general correlated channels, some limiting upper bounds for the first three moments are also provided. For uncorrelated channels and correlated channels satisfying certain conditions, it is proved that T converges to a Normal random variable. A Gamma distribution and a generalized Gamma distribution are proposed as approximations to the finite sample distribution of T. Simulations suggest that these approximate distributions can be used to estimate accurately the probability of errors even for very small dimensions (e.g., two transmit antennas).

/pdf/on-the-distribution-of-sinr-for-the-mmse-mimo-receiver-and-5g9bzw028d.pdf

On the distribution of SINR for the MMSE MIMO receiver and performance analysis

We study the problem of estimating the leading eigenvectors of a high-dimensional population covariance matrix based on independent Gaussian observations. We establish a lower bound on the minimax risk of estimators under the l2 loss, in the joint limit as dimension and sample size increase to infinity, under various models of sparsity for the population eigenvectors. The lower bound on the risk points to the existence of different regimes of sparsity of the eigenvectors. We also propose a new method for estimating the eigenvectors by a two-stage coordinate selection scheme.

/pdf/minimax-bounds-for-sparse-pca-with-noisy-high-dimensional-1q6x2m7u87.pdf

Minimax bounds for sparse PCA with noisy high-dimensional data

https://escholarship.org/content/qt2350x9hz/qt2350x9hz.pdf

Debashis Paul

Papers

Prediction by supervised principal components

Asymptotics of sample eigenstructure for a large dimensional spiked covariance model

On the distribution of SINR for the MMSE MIMO receiver and performance analysis

Minimax bounds for sparse PCA with noisy high-dimensional data

Random matrix theory in statistics: A review