A full-rank matrix ${\bf A}\in \mathbb{R}^{n\times m}$ with $n<m$ generates an underdetermined system of linear equations ${\bf Ax} = {\bf b}$ having infinitely many solutions. Suppose we seek the sparsest solution, i.e., the one with the fewest nonzero entries. Can it ever be unique? If so, when? As optimization of sparsity is combinatorial in nature, are there efficient methods for finding the sparsest solution? These questions have been answered positively and constructively in recent years, exposing a wide variety of surprising phenomena, in particular the existence of easily verifiable conditions under which optimally sparse solutions can be found by concrete, effective computational methods.

Such theoretical results inspire a bold perspective on some important practical problems in signal and image processing. Several well-known signal and image processing problems can be cast as demanding solutions of undetermined systems of equations. Such problems have previously seemed, to many, intractable, but there is considerable evidence that these problems often have sparse solutions. Hence, advances in finding sparse solutions to underdetermined systems have energized research on such signal and image processing problems—to striking effect.

In this paper we review the theoretical results on sparse solutions of linear systems, empirical results on sparse modeling of signals and images, and recent applications in inverse problems and compression in image processing. This work lies at the intersection of signal processing and applied mathematics, and arose initially from the wavelets and harmonic analysis research communities. The aim of this paper is to introduce a few key notions and applications connected to sparsity, targeting newcomers interested in either the mathematical aspects of this area or its applications.

background

/pdf/from-sparse-solutions-of-systems-of-equations-to-sparse-1y2kv866zu.pdf

Oriented Matroids

From Sparse Solutions of Systems of Equations to Sparse Modeling of Signals and Images

We construct Markov chain algorithms for sampling from discrete exponential families conditional on a sufficient statistic. Examples include contingency tables, logistic regression, and spectral analysis of permutation data. The algorithms involve computations in polynomial rings using Grobner bases.

/pdf/algebraic-algorithms-for-sampling-from-conditional-30s3b9fbtm.pdf

Algebraic algorithms for sampling from conditional distributions

This is a direct continuation of Shellable Nonpure Complexes and Posets. I, which appeared in Transactions of the American Mathematical Society 348 (1996), 1299-1327. 8. Interval-generated lattices and dominance order In this section and the following one we will continue exemplifying the applicability of lexicographic shellability to nonpure posets. Let F = {I1, I2, . . . , In} be a family of intervals of integers, by which is meant sets of the form [a, b] = {a, a + 1, . . . , b}, a ≤ b. We assume that there are no containments among these intervals, and that they are ordered so that their left and right endpoints are increasing. Let L(F) be the lattice of all sets that are unions of subfamilies of F , ordered by inclusion. Such interval-generated lattices L(F) were introduced and studied by Greene [G]. Define an edge-labeling λ of L(F) as follows. If A → B is a covering and a = max(B \A), then λ(A→ B) = { −a, if (a+ 1) ∈ A and a is the left endpoint of some I ∈ F ,

/pdf/shellable-nonpure-complexes-and-posets-ii-7mthqkp11j.pdf

Shellable nonpure complexes and posets. II

Euclidean distance geometry is the study of Euclidean geometry based on the concept of distance. This is useful in several applications where the input data consist of an incomplete set of distances and the output is a set of points in Euclidean space realizing those given distances. We survey the theory of Euclidean distance geometry and its most important applications, with special emphasis on molecular conformation problems.

/pdf/euclidean-distance-geometry-and-applications-bzz54wgm3y.pdf

Euclidean Distance Geometry and Applications

We abstract out the core search problem of active learning schemes, to better understand the extent to which adaptive labeling can improve sample complexity. We give various upper and lower bounds on the number of labels which need to be queried, and we prove that a popular greedy active learning rule is approximately as good as any other strategy for minimizing this number of labels.

/pdf/analysis-of-a-greedy-active-learning-strategy-2201oj9lfv.pdf

Oriented Matroids

Citations

From Sparse Solutions of Systems of Equations to Sparse Modeling of Signals and Images

Algebraic algorithms for sampling from conditional distributions

Shellable nonpure complexes and posets. II

Euclidean Distance Geometry and Applications

Analysis of a greedy active learning strategy

Related Papers (5)

Lectures on Polytopes

Arrangements of hyperplanes

Encyclopedia of Mathematics and its Applications.

Arrangements and Spreads

Enumerative Combinatorics