This article is about a curious phenomenon. Suppose we have a data matrix, which is the superposition of a low-rank component and a sparse component. Can we recover each component individuallyq We prove that under some suitable assumptions, it is possible to recover both the low-rank and the sparse components exactly by solving a very convenient convex program called Principal Component Pursuit; among all feasible decompositions, simply minimize a weighted combination of the nuclear norm and of the e1 norm. This suggests the possibility of a principled approach to robust principal component analysis since our methodology and results assert that one can recover the principal components of a data matrix even though a positive fraction of its entries are arbitrarily corrupted. This extends to the situation where a fraction of the entries are missing as well. We discuss an algorithm for solving this optimization problem, and present applications in the area of video surveillance, where our methodology allows for the detection of objects in a cluttered background, and in the area of face recognition, where it offers a principled way of removing shadows and specularities in images of faces.

/pdf/robust-principal-component-analysis-59kf1xh72d.pdf

Robust principal component analysis

Convergence of Probability Measures. By P. Billingsley. Chichester, Sussex, Wiley, 1968. xii, 253 p. 9 1/4“. 117s.

Convergence of Probability Measures

Large datasets are increasingly common and are often difficult to interpret. Principal component analysis (PCA) is a technique for reducing the dimensionality of such datasets, increasing interpretability but at the same time minimizing information loss. It does so by creating new uncorrelated variables that successively maximize variance. Finding such new variables, the principal components, reduces to solving an eigenvalue/eigenvector problem, and the new variables are defined by the dataset at hand, not a priori , hence making PCA an adaptive data analysis technique. It is adaptive in another sense too, since variants of the technique have been developed that are tailored to various different data types and structures. This article will begin by introducing the basic ideas of PCA, discussing what it can and cannot do. It will then describe some variants of PCA and their application.

https://royalsocietypublishing.org/doi/pdf/10.1098/rsta.2015.0202

Principal component analysis: a review and recent developments

Point defects and impurities strongly affect the physical properties of materials and have a decisive impact on their performance in applications. First-principles calculations have emerged as a powerful approach that complements experiments and can serve as a predictive tool in the identification and characterization of defects. The theoretical modeling of point defects in crystalline materials by means of electronic-structure calculations, with an emphasis on approaches based on density functional theory (DFT), is reviewed. A general thermodynamic formalism is laid down to investigate the physical properties of point defects independent of the materials class (semiconductors, insulators, and metals), indicating how the relevant thermodynamic quantities, such as formation energy, entropy, and excess volume, can be obtained from electronic structure calculations. Practical aspects such as the supercell approach and efficient strategies to extrapolate to the isolated-defect or dilute limit are discussed. Recent advances in tractable approximations to the exchange-correlation functional ($\mathrm{DFT}+U$, hybrid functionals) and approaches beyond DFT are highlighted. These advances have largely removed the long-standing uncertainty of defect formation energies in semiconductors and insulators due to the failure of standard DFT to reproduce band gaps. Two case studies illustrate how such calculations provide new insight into the physics and role of point defects in real materials.

First-principles calculations for point defects in solids

SUMMARY This paper investigates the properties of a semiparametric method for estimating the dependence parameters in a family of multivariate distributions. The proposed estimator, obtained as a solution of a pseudo-likelihood equation, is shown to be consistent, asymptotically normal and fully efficient at independence. A natural estimator of its asymptotic variance is proved to be consistent. Comparisons are made with alternative semiparametric estimators in the special case of Clayton's model for association in bivariate data.

https://www.jstor.org/stable/pdfplus/2337532.pdf

A semiparametric estimation procedure of dependence parameters in multivariate families of distributions

In statistical errors in variables models one observes a noisy convolution of an unknown density with a known error distribution, and the problem is to recover the unknown density. This problem amounts to ill-posed inversion of an operator. Such inversion requires regularization. After a brief summary of regularization in general we turn to deconvolution of functions on Abelian groups and focus in particular on the circle. Finally we propose a kind of error in variables model on the sphere (see also Chang (1989)) and mention some ensuing problems.

Regularized Deconvolution on the Circle and the Sphere

During the past the convergence analysis for linear statistical inverse problems has mainly focused on spectral cut-off and Tikhonov type estimators. Spectral cut-off estimators achieve minimax rates for a broad range of smoothness classes and operators, but their practical usefulness is limited by the fact that they require a complete spectral decomposition of the operator. Tikhonov estimators are simpler to compute, but still involve the inversion of an operator and achieve minimax rates only in restricted smoothness classes. In this paper we introduce a unifying technique to study the mean square error of a large class of regularization methods (spectral methods) including the aforementioned estimators as well as many iterative methods, such as i-methods and the Landweber iteration. The latter estimators converge at the same rate as spectral cut-off, but only require matrixvector products. Our results are applied to various problems, in particular we obtain precise convergence rates for satellite gradiometry, L2-boosting, and errors in variable problems.

Convergence rates of general regularization methods for statistical inverse problems and applications

Some properties of canonical correlations and variates in infinite dimensions

In this paper, the asymptotic distributions of estimators for the regularized functional canonical correlation and variates of the population are derived. The method is based on the possibility of expressing these regularized quantities as the maximum eigenvalue and the corresponding eigenfunctions of an associated pair of regularized operators, similar to the Euclidean case. The known weak convergence of the sample covariance operator, coupled with a delta-method for analytic functions of covariance operators, yields the weak convergence of the pair of associated operators. From the latter weak convergence, the limiting distributions of the canonical quantities of interest can be derived with the help of some further perturbation theory.

The delta method for analytic functions of random operators with application to functional data

Inverse estimation is concerned with estimation of a signal, where the signal is indirectly observed in the presence of random noise. By indirect we mean, more precisely, that we observe a transform of the signal. The transforms that we consider here are quite general and in particular not restricted to compact operators. This general abstract setup covers many examples of practical interest. One possible approach to signal recovery is to apply a regularized inverse of the transform to the observed image. Although optimal rates for the smoothing parameter when the number of data points increases indefinitely can be established, this does not give any information about a suitable choice of the parameter for a fixed given data set. In order to solve that problem we propose a cross-validation method that can even be formulated in the general abstract case, where the expression is given in the "spectral domain". In the special instances of indirect density and regression estimation, moreover, an equivalent expression in the "time domain" can be given. A simulation study is devoted to a deconvolution problem with quite satisfactory results.

Frits H. Ruymgaart

Papers

Regularized Deconvolution on the Circle and the Sphere

Convergence rates of general regularization methods for statistical inverse problems and applications

Some properties of canonical correlations and variates in infinite dimensions

The delta method for analytic functions of random operators with application to functional data

Cross-validation for Parameter Selection in Inverse Estimation Problems