Consider an underdetermined system of linear equations y = Ax with known y and d × n matrix A. We seek the nonnegative x with the fewest nonzeros satisfying y = Ax. In general, this problem is NP-hard. However, for many matrices A there is a threshold phenomenon: if the sparsest solution is sufficiently sparse, it can be found by linear programming. We explain this by the theory of convex polytopes. Let aj denote the jth column of A, 1 ≤ j ≤ n, let a0 = 0 and P denote the convex hull of the aj. We say the polytope P is outwardly k-neighborly if every subset of k vertices not including 0 spans a face of P. We show that outward k-neighborliness is equivalent to the statement that, whenever y = Ax has a nonnegative solution with at most k nonzeros, it is the nonnegative solution to y = Ax having minimal sum. We also consider weak neighborliness, where the overwhelming majority of k-sets of ajs not containing 0 span a face of P. This implies that most nonnegative vectors x with k nonzeros are uniquely recoverable from y = Ax by linear programming. Numerous corollaries follow by invoking neighborliness results. For example, for most large n by 2n underdetermined systems having a solution with fewer nonzeros than roughly half the number of equations, the sparsest solution can be found by linear programming.

/pdf/sparse-nonnegative-solution-of-underdetermined-linear-1s341gbazq.pdf

Sparse nonnegative solution of underdetermined linear equations by linear programming

1.1. Three surprises of high dimensions. This paper develops asymptotic methods to count faces of random high-dimensional polytopes; a seemingly dry and unpromising pursuit. Yet our conclusions have surprising implications - in statistics, probability, information theory, and signal processing - with potential impacts in practical subjects like medical imaging and digital communications. Before involving the reader in our lengthy analysis of high-dimensional face counting, we describe three implications of our results. 1.1.1. Convex Hulls of Gaussian Point Clouds. Consider a random point cloud of n points xi, i = 1, . . . , n, sampled independently and identically from a Gaussian distribution in R d with nonsingular covariance. This is a standard model of multivariate data; its properties are increasingly important in a wide range of applications. At the same time, it is an attractive and in some sense timeless object for theoretical study. Properties of the convex hull of the random point cloud X = {xi} have attracted interest for several decades, increasingly so in recent years; there is a nowvoluminous literature on the subject. The results could be significant for understanding outlier detection, or classification problems in machine learning.

Counting faces of randomly-projected polytopes when the projection radically lowers dimension

The purpose of these lecture notes is to provide an introduction to the general theory of empirical risk minimization with an emphasis on excess risk bounds and oracle inequalities in penalized problems. In recent years, there have been new developments in this area motivated by the study of new classes of methods in machine learning such as large margin classification methods (boosting, kernel machines). The main probabilistic tools involved in the analysis of these problems are concentration and deviation inequalities by Talagrand along with other methods of empirical processes theory (symmetrization inequalities, contraction inequality for Rademacher sums, entropy and generic chaining bounds). Sparse recovery based on l_1-type penalization and low rank matrix recovery based on the nuclear norm penalization are other active areas of research, where the main problems can be stated in the framework of penalized empirical risk minimization, and concentration inequalities and empirical processes tools have proved to be very useful.

Oracle Inequalities in Empirical Risk Minimization and Sparse Recovery Problems: École d'Été de Probabilités de Saint-Flour XXXVIII-2008

Let A be a d x n matrix and T = T(n-1) be the standard simplex in Rn. Suppose that d and n are both large and comparable: d approximately deltan, delta in (0, 1). We count the faces of the projected simplex AT when the projector A is chosen uniformly at random from the Grassmann manifold of d-dimensional orthoprojectors of Rn. We derive rhoN(delta) > 0 with the property that, for any rho or = f(k)(T)(1-epsilon). Vershik and Sporyshev previously showed existence of a threshold rhoVS(delta) > 0 at which phase transition occurs in k/d. We compute and display rhoVS and compare with rhoN. Corollaries are as follows. (1) The convex hull of n Gaussian samples in Rd, with n large and proportional to d, has the same k-skeleton as the (n-1) simplex, for k < rhoN (d/n)d(1 + oP(1)). (2) There is a "phase transition" in the ability of linear programming to find the sparsest nonnegative solution to systems of underdetermined linear equations. For most systems having a solution with fewer than rhoVS(d/n)d(1 + o(1)) nonzeros, linear programming will find that solution.

/pdf/neighborliness-of-randomly-projected-simplices-in-high-320g0f6pg9.pdf

Neighborliness of randomly-projected simplices in high dimensions

Recent research indicates that many convex optimization problems with random constraints exhibit a
phase transition as the number of constraints increases. For example, this phenomenon emerges in the
 l_1 minimization method for identifying a sparse vector from random linear measurements. Indeed, the
 l_1 approach succeeds with high probability when the number of measurements exceeds a threshold that
depends on the sparsity level; otherwise, it fails with high probability. This paper provides the first rigorous
analysis that explains why phase transitions are ubiquitous in random convex optimization problems.
It also describes tools for making reliable predictions about the quantitative aspects of the transition,
including the location and the width of the transition region. These techniques apply to regularized linear
inverse problems with random measurements, to demixing problems under a random incoherence model,
and also to cone programs with random affine constraints. The applied results depend on foundational
research in conic geometry. This paper introduces a summary parameter, called the statistical dimension,
that canonically extends the dimension of a linear subspace to the class of convex cones. The main technical
result demonstrates that the sequence of intrinsic volumes of a convex cone concentrates sharply
around the statistical dimension. This fact leads to accurate bounds on the probability that a randomly
rotated cone shares a ray with a fixed cone.

/pdf/living-on-the-edge-phase-transitions-in-convex-programs-with-5fgd0l8ora.pdf

Living on the edge: phase transitions in convex programs with random data

Precise asymptotic formulae are obtained for the expected number ofk-faces of the orthogonal projection of a regularn-simplex inn-space onto a randomly chosen isotropic subspace of fixed dimension or codimension, as the dimensionn tends to infinity.

/pdf/random-projections-of-regular-simplices-5sy32il5oz.pdf

Random projections of regular simplices

Denote the expected number of facets and vertices and the expected volume of the convex hullPn ofn random points, selected independently and uniformly from the interior of a simpled-polytope byEn(f), En(v), andEn(V), respectively. In this note we determine the sharp constants of the asymptotic expansion ofEn(f), En(v), andEn(V), asn tends to infinity. Further, some results concerning the expected shape ofPn are given.

/pdf/on-the-convex-hull-of-uniform-random-points-in-a-simpled-6f8a42k3f2.pdf

Fernando Affentranger

Papers

Random projections of regular simplices

On the convex hull of uniform random points in a simpled-polytope

Random spheres in a convex body