Large datasets are increasingly common and are often difficult to interpret. Principal component analysis (PCA) is a technique for reducing the dimensionality of such datasets, increasing interpretability but at the same time minimizing information loss. It does so by creating new uncorrelated variables that successively maximize variance. Finding such new variables, the principal components, reduces to solving an eigenvalue/eigenvector problem, and the new variables are defined by the dataset at hand, not a priori , hence making PCA an adaptive data analysis technique. It is adaptive in another sense too, since variants of the technique have been developed that are tailored to various different data types and structures. This article will begin by introducing the basic ideas of PCA, discussing what it can and cannot do. It will then describe some variants of PCA and their application.

https://royalsocietypublishing.org/doi/pdf/10.1098/rsta.2015.0202

Principal component analysis: a review and recent developments

Independent component models have gained increasing interest in various fields of applications in recent years. The basic independent component model is a semiparametric model assuming that a p-variate observed random vector is a linear transformation of an unobserved vector of p independent latent variables. This linear transformation is given by an unknown mixing matrix, and one of the main objectives of independent component analysis (ICA) is to estimate an unmixing matrix by means of which the latent variables can be recovered. In this article, we discuss the basic independent component model in detail, define the concepts and analysis tools carefully, and consider two families of ICA estimates. The statistical properties (consistency, asymptotic normality, efficiency, robustness) of the estimates can be analyzed and compared via the so called gain matrices. Some extensions of the basic independent component model, such as models with additive noise or models with dependent observations, are briefly discussed. The article ends with a short example.


Keywords:

blind source separation;
fastICA;
independent component model;
independent subspace analysis;
mixing matrix;
overcomplete ICA;
undercomplete ICA;
unmixing matrix

Independent Component Analysis

Independent Component Analysis.

Matrix Differential Calculus with Applications in Statistics and Econometrics

Probability and Measure

A new multivariate concept of quantile, based on a directional version of Koenker and Bassett’s traditional regression quantiles, is introduced for multivariate location and multiple-output regression problems. In their empirical version, those quantiles can be computed efficiently via linear programming techniques. Consistency, Bahadur representation and asymptotic normality results are established. Most importantly, the contours generated by those quantiles are shown to coincide with the classical halfspace depth contours associated with the name of Tukey. This relation does not only allow for efficient depth contour computations by means of parametric linear programming, but also for transferring from the quantile to the depth universe such asymptotic results as Bahadur representations. Finally, linear programming duality opens the way to promising developments in depth-related multivariate rank-based inference.

/pdf/multivariate-quantiles-and-multiple-output-regression-1vqnstd83h.pdf

Multivariate quantiles and multiple-output regression quantiles: From L1 optimization to halfspace depth

We propose a family of tests, based on Randles' (1989) concept of interdirections and the ranks of pseudo-Mahalanobis distances computed with respect to a multivariate M-estimator of scatter due to Tyler (1987), for the multivariate one-sample problem under elliptical symmetry. These tests, which generalize the univariate signed-rank tests, are affine-invariant. Depending on the score function considered (van der Waerden, Laplace,...), they allow for locally asymptotically maximin tests at selected densities (multivariate normal, multivariate double-exponential,...). Local powers and asymptotic relative efficiencies are derived--with respect to Hotelling's test, Randles' (1989) multivariate sign test, Peters and Randles' (1990) Wilcoxon-type test, and with respect to the Oja median tests. We, moreover, extend to the multivariate setting two famous univariate results: the traditional Chernoff-Savage (1958) property, showing that Hotelling's traditional procedure is uniformly dominated, in the Pitman sense, by the van der Waerden version of our tests, and the celebrated Hodges-Lehmann (1956) ".864 result," providing, for any fixed space dimension $k$, the lower bound for the asymptotic relative efficiency of Wilcoxon-type tests with respect to Hotelling's. These asymptotic results are confirmed by a Monte Carlo investigation, and application to a real data set.

/pdf/optimal-tests-for-multivariate-location-based-on-46q70bld0m.pdf

Optimal tests for multivariate location based on interdirections and pseudo-Mahalanobis ranks

We propose a class of rank-based procedures for testing that the shape matrix $\mathbf{V}$ of an elliptical distribution (with unspecified center of symmetry, scale and radial density) has some fixed value ${\mathbf{V}}_0$; this includes, for ${\mathbf{V}}_0={\mathbf{I}}_k$, the problem of testing for sphericity as an important particular case. The proposed tests are invariant under translations, monotone radial transformations, rotations and reflections with respect to the estimated center of symmetry. They are valid without any moment assumption. For adequately chosen scores, they are locally asymptotically maximin (in the Le Cam sense) at given radial densities. They are strictly distribution-free when the center of symmetry is specified, and asymptotically so when it must be estimated. The multivariate ranks used throughout are those of the distances--in the metric associated with the null value ${\mathbf{V}}_0$ of the shape matrix--between the observations and the (estimated) center of the distribution. Local powers (against elliptical alternatives) and asymptotic relative efficiencies (AREs) are derived with respect to the adjusted Mauchly test (a modified version of the Gaussian likelihood ratio procedure proposed by Muirhead and Waternaux [Biometrika 67 (1980) 31--43]) or, equivalently, with respect to (an extension of) the test for sphericity introduced by John [Biometrika 58 (1971) 169--174]. For Gaussian scores, these AREs are uniformly larger than one, irrespective of the actual radial density. Necessary and/or sufficient conditions for consistency under nonlocal, possibly nonelliptical alternatives are given. Finite sample performances are investigated via a Monte Carlo study.

Semiparametrically efficient rank-based inference for shape I. optimal rank-based tests for sphericity

A new multivariate concept of quantile, based on a directional version of Koenker and Bassett's traditional regression quantiles, is introduced for multivariate location and multiple-output regression problems. In their empirical version, those quantiles can be computed efficiently via linear programming techniques. Consistency, Bahadur representation and asymptotic normality results are established. Most importantly, the contours generated by those quantiles are shown to coincide with the classical halfspace depth contours associated with the name of Tukey. This relation does not only allow for efficient depth contour computations by means of parametric linear programming, but also for transferring from the quantile to the depth universe such asymptotic results as Bahadur representations. Finally, linear programming duality opens the way to promising developments in depth-related multivariate rank-based inference.

Multivariate quantiles and multiple-output regression quantiles: From $L_1$ optimization to halfspace depth

http://homepages.ulb.ac.be/~dpaindav/stuff/Articles/Article23/statprob.pdf

Davy Paindaveine

Papers

Multivariate quantiles and multiple-output regression quantiles: From L1 optimization to halfspace depth

Optimal tests for multivariate location based on interdirections and pseudo-Mahalanobis ranks

Semiparametrically efficient rank-based inference for shape I. optimal rank-based tests for sphericity

Multivariate quantiles and multiple-output regression quantiles: From $L_1$ optimization to halfspace depth

A canonical definition of shape