Showing papers in "Foundations of Computational Mathematics in 2009"
TL;DR: It is proved that one can perfectly recover most low-rank matrices from what appears to be an incomplete set of entries, and that objects other than signals and images can be perfectly reconstructed from very limited information.
Abstract: We consider a problem of considerable practical interest: the recovery of a data matrix from a sampling of its entries. Suppose that we observe m entries selected uniformly at random from a matrix M. Can we complete the matrix and recover the entries that we have not seen?
We show that one can perfectly recover most low-rank matrices from what appears to be an incomplete set of entries. We prove that if the number m of sampled entries obeys $$m\ge C\,n^{1.2}r\log n$$ for some positive numerical constant C, then with very high probability, most n×n matrices of rank r can be perfectly recovered by solving a simple convex optimization program. This program finds the matrix with minimum nuclear norm that fits the data. The condition above assumes that the rank is not too large. However, if one replaces the 1.2 exponent with 1.25, then the result holds for all values of the rank. Similar results hold for arbitrary rectangular matrices as well. Our results are connected with the recent literature on compressed sensing, and show that objects other than signals and images can be perfectly reconstructed from very limited information.
5,274 citations
TL;DR: This paper finds a simple regularized version of Orthogonal Matching Pursuit (ROMP) which has advantages of both approaches: the speed and transparency of OMP and the strong uniform guarantees of L1-minimization.
Abstract: This paper seeks to bridge the two major algorithmic approaches to sparse signal recovery from an incomplete set of linear measurements—L1-minimization methods and iterative methods (Matching Pursuits). We find a simple regularized version of Orthogonal Matching Pursuit (ROMP) which has advantages of both approaches: the speed and transparency of OMP and the strong uniform guarantees of L1-minimization. Our algorithm, ROMP, reconstructs a sparse signal in a number of iterations linear in the sparsity, and the reconstruction is exact provided the linear measurements satisfy the uniform uncertainty principle.
998 citations
TL;DR: A new approach for nonadaptive dimensionality reduction of manifold-modeled data is proposed, demonstrating that a small number of random linear projections can preserve key information about a manifold- modeled signal.
Abstract: We propose a new approach for nonadaptive dimensionality reduction of manifold-modeled data, demonstrating that a small number of random linear projections can preserve key information about a manifold-modeled signal. We center our analysis on the effect of a random linear projection operator Φ:ℝ N →ℝM , M
488 citations
TL;DR: It is proved that von Neumann’s method of “alternating projections” converges locally to a point in the intersection, at a linear rate associated with a modulus of regularity.
Abstract: The idea of a finite collection of closed sets having “linearly regular intersection” at a point is crucial in variational analysis. This central theoretical condition also has striking algorithmic consequences: in the case of two sets, one of which satisfies a further regularity condition (convexity or smoothness, for example), we prove that von Neumann’s method of “alternating projections” converges locally to a point in the intersection, at a linear rate associated with a modulus of regularity. As a consequence, in the case of several arbitrary closed sets having linearly regular intersection at some point, the method of “averaged projections” converges locally at a linear rate to a point in the intersection. Inexact versions of both algorithms also converge linearly.
253 citations
TL;DR: An algebraic formulation is given that extends persistence to essential homology for any filtered space, an algorithm is presented to calculate it, and how it aids the ability to recognize shape features for codimension 1 submanifolds of Euclidean space is described.
Abstract: Persistent homology has proven to be a useful tool in a variety of contexts, including the recognition and measurement of shape characteristics of surfaces in ℝ3. Persistence pairs homology classes that are born and die in a filtration of a topological space, but does not pair its actual homology classes. For the sublevelset filtration of a surface in ℝ3, persistence has been extended to a pairing of essential classes using Reeb graphs. In this paper, we give an algebraic formulation that extends persistence to essential homology for any filtered space, present an algorithm to calculate it, and describe how it aids our ability to recognize shape features for codimension 1 submanifolds of Euclidean space. The extension derives from Poincare duality but generalizes to nonmanifold spaces. We prove stability for general triangulated spaces and duality as well as symmetry for triangulated manifolds.
198 citations
TL;DR: Numerical analysis of structure-preserving time-integrators for rigid body-type mechanical systems are derived from a discrete Hamilton–Pontryagin variational principle and a novel class of variational partitioned Runge–Kutta methods on Lie groups are derived.
Abstract: In this paper, structure-preserving time-integrators for rigid body-type mechanical systems are derived from a discrete Hamilton–Pontryagin variational principle. From this principle, one can derive a novel class of variational partitioned Runge–Kutta methods on Lie groups. Included among these integrators are generalizations of symplectic Euler and Stormer–Verlet integrators from flat spaces to Lie groups. Because of their variational design, these integrators preserve a discrete momentum map (in the presence of symmetry) and a symplectic form.
In a companion paper, we perform a numerical analysis of these methods and report on numerical experiments on the rigid body and chaotic dynamics of an underwater vehicle. The numerics reveal that these variational integrators possess structure-preserving properties that methods designed to preserve momentum (using the coadjoint action of the Lie group) and energy (for example, by projection) lack.
160 citations
TL;DR: Numerical integration of Lipschitz functionals on a Banach space by means of deterministic and randomized (Monte Carlo) algorithms is studied and asymptotic behavior of quantization numbers and Kolmogorov widths for diffusion processes is determined.
Abstract: We study numerical integration of Lipschitz functionals on a Banach space by means of deterministic and randomized (Monte Carlo) algorithms. This quadrature problem is shown to be closely related to the problem of quantization and to the average Kolmogorov widths of the underlying probability measure. In addition to the general setting, we analyze, in particular, integration with respect to Gaussian measures and distributions of diffusion processes. We derive lower bounds for the worst case error of every algorithm in terms of its cost, and we present matching upper bounds, up to logarithms, and corresponding almost optimal algorithms. As auxiliary results, we determine the asymptotic behavior of quantization numbers and Kolmogorov widths for diffusion processes.
94 citations
TL;DR: Theoretical Spectral Curvature Clustering (TSCC) as discussed by the authors is a combination of Govindu's multi-way spectral clustering framework (CVPR 2005) and Ng et al.
Abstract: The problem of Hybrid Linear Modeling (HLM) is to model and segment data using a mixture of affine subspaces. Different strategies have been proposed to solve this problem, however, rigorous analysis justifying their performance is missing. This paper suggests the Theoretical Spectral Curvature Clustering (TSCC) algorithm for solving the HLM problem and provides careful analysis to justify it. The TSCC algorithm is practically a combination of Govindu’s multi-way spectral clustering framework (CVPR 2005) and Ng et al.’s spectral clustering algorithm (NIPS 2001). The main result of this paper states that if the given data is sampled from a mixture of distributions concentrated around affine subspaces, then with high sampling probability the TSCC algorithm segments well the different underlying clusters. The goodness of clustering depends on the within-cluster errors, the between-clusters interaction, and a tuning parameter applied by TSCC. The proof also provides new insights for the analysis of Ng et al. (NIPS 2001).
72 citations
TL;DR: In this article, a probabilistic symbolic algorithm for solving zero-dimensional sparse systems is presented, which combines a symbolic homotopy procedure, based on a flat deformation of a certain morphism of affine varieties, with the polyhedral deformation for Huber and Sturmfels.
Abstract: We exhibit a probabilistic symbolic algorithm for solving zero-dimensional sparse systems. Our algorithm combines a symbolic homotopy procedure, based on a flat deformation of a certain morphism of affine varieties, with the polyhedral deformation of Huber and Sturmfels. The complexity of our algorithm is cubic in the size of the combinatorial structure of the input system. This size is mainly represented by the cardinality and mixed volume of Newton polytopes of the input polynomials and an arithmetic analogue of the mixed volume associated to the deformations under consideration.
65 citations
TL;DR: In this article, the authors provide a recursive formula for the logarithm of the solutions of the equations X = 1+λ a ≺ X and Y = 1−λ Y ≻ a in A[[λ]] where (A,≺,≻) is a dendriform algebra.
Abstract: We provide a refined approach to the classical Magnus (Commun. Pure Appl. Math. 7:649–673, [1954]) and Fer expansion (Bull. Classe Sci. Acad. R. Belg. 44:818–829, [1958]), unveiling a new structure by using the language of dendriform and pre-Lie algebras. The recursive formula for the logarithm of the solutions of the equations X=1+λ a ≺ X and Y=1−λ Y ≻ a in A[[λ]] is provided, where (A,≺,≻) is a dendriform algebra. Then we present the solutions to these equations as an infinite product expansion of exponentials. Both formulae involve the pre-Lie product naturally associated with the dendriform structure. Several applications are presented.
61 citations
TL;DR: In this paper, a new complexity measure of a path of (problems, solutions) pairs in terms of the length of the path in the condition metric is introduced. But the complexity measure is not applicable to the case of geodesic paths.
Abstract: We introduce a new complexity measure of a path of (problems, solutions) pairs in terms of the length of the path in the condition metric which we define in the article. The measure gives an upper bound for the number of Newton steps sufficient to approximate the path discretely starting from one end and thus produce an approximate zero for the endpoint. This motivates the study of short paths or geodesics in the condition metric.
TL;DR: A reduction theorem is proved which states that the canonical variables on the Lie group can be eliminated, if and only if the velocity map is a Lie algebra action, thereby producing the Euler–Poincaré (EP) equation for the vector space variables.
Abstract: The Clebsch method provides a unifying approach for deriving variational principles for continuous and discrete dynamical systems where elements of a vector space are used to control dynamics on the cotangent bundle of a Lie group via a velocity map. This paper proves a reduction theorem which states that the canonical variables on the Lie group can be eliminated, if and only if the velocity map is a Lie algebra action, thereby producing the Euler–Poincare (EP) equation for the vector space variables. In this case, the map from the canonical variables on the Lie group to the vector space is the standard momentum map defined using the diamond operator. We apply the Clebsch method in examples of the rotating rigid body and the incompressible Euler equations. Along the way, we explain how singular solutions of the EP equation for the diffeomorphism group (EPDiff) arise as momentum maps in the Clebsch approach. In the case of finite-dimensional Lie groups, the Clebsch variational principle is discretized to produce a variational integrator for the dynamical system. We obtain a discrete map from which the variables on the cotangent bundle of a Lie group may be eliminated to produce a discrete EP equation for elements of the vector space. We give an integrator for the rotating rigid body as an example. We also briefly discuss how to discretize infinite-dimensional Clebsch systems, so as to produce conservative numerical methods for fluid dynamics.
TL;DR: An error analysis for SVMR is provides an explicit learning rate for the SVMR algorithm under some assumptions, and some recently developed methods for analysis of classification algorithms such as the projection operator and the iteration technique are introduced.
Abstract: Support vector machines regression (SVMR) is a regularized learning algorithm in reproducing kernel Hilbert spaces with a loss function called the e-insensitive loss function. Compared with the well-understood least square regression, the study of SVMR is not satisfactory, especially the quantitative estimates of the convergence of this algorithm. This paper provides an error analysis for SVMR, and introduces some recently developed methods for analysis of classification algorithms such as the projection operator and the iteration technique. The main result is an explicit learning rate for the SVMR algorithm under some assumptions.
TL;DR: It is proved that given the two pairs (fi,ζi), i=1,2, there exist a short path joining them such that the complexity of following the path is bounded by the logarithm of the condition number of the problems.
Abstract: We study geometric properties of the solution variety for the problem of approximating solutions of systems of polynomial equations. We prove that given the two pairs (f i ,ζ i ), i=1,2, there exist a short path joining them such that the complexity of following the path is bounded by the logarithm of the condition number of the problems.
TL;DR: It is shown that a redundancy-free sufficient set of integrability conditions can be constructed in a time proportional to the number of equations cubed.
Abstract: Every orthonomic system of partial differential equations is known to possess a finite number of integrability conditions sufficient to ensure the validity of them all. Here we show that a redundancy-free sufficient set of integrability conditions can be constructed in a time proportional to the number of equations cubed.
TL;DR: In this paper, the Lyapunov-Schmidt reduction was used to obtain rigorous bounds for the Poincare map of the Rossler system, showing the existence of two period-doubling bifurcations and a branch of period two points connecting them.
Abstract: Using rigorous numerical methods, we validate a part of the bifurcation diagram for a Poincare map of the Rossler system (Rossler in Phys. Lett. A 57(5):397–398, 1976)—the existence of two period-doubling bifurcations and the existence of a branch of period two points connecting them. Our approach is based on the Lyapunov–Schmidt reduction and uses the C r -Lohner algorithm (Wilczak and Zgliczynski, available at http://www.ii.uj.edu.pl/~wilczak) to obtain rigorous bounds for the Rossler system.
TL;DR: It turns out that infinitesimal and generic quantifiers can be eliminated, so that the relevant complexity classes can be described in terms of the usual quantifiers only.
Abstract: We define new complexity classes in the Blum–Shub–Smale theory of computation over the reals, in the spirit of the polynomial hierarchy, with the help of infinitesimal and generic quantifiers. Basic topological properties of semialgebraic sets like boundedness, closedness, compactness, as well as the continuity of semialgebraic functions are shown to be complete in these new classes. All attempts to classify the complexity of these problems in terms of the previously studied complexity classes have failed. We also obtain completeness results in the Turing model for the corresponding discrete problems. In this setting, it turns out that infinitesimal and generic quantifiers can be eliminated, so that the relevant complexity classes can be described in terms of the usual quantifiers only.
TL;DR: In this paper, the authors show that if the support of a measure is an analytic Jordan arc and if the measure itself is absolutely continuous with respect to the equilibrium distribution of that arc with Dini-smooth nonvanishing density, then the diagonal multipoint Pade approximants associated with appropriate interpolation schemes converge locally uniformly to the approximated Cauchy transform in the complement of the arc.
Abstract: We consider multipoint Pade approximation to Cauchy transforms of complex measures. We show that if the support of a measure is an analytic Jordan arc and if the measure itself is absolutely continuous with respect to the equilibrium distribution of that arc with Dini-smooth nonvanishing density, then the diagonal multipoint Pade approximants associated with appropriate interpolation schemes converge locally uniformly to the approximated Cauchy transform in the complement of the arc. This asymptotic behavior of Pade approximants is deduced from the analysis of underlying non-Hermitian orthogonal polynomials, for which we use classical properties of Hankel and Toeplitz operators on smooth curves. A construction of the appropriate interpolation schemes is explicit granted the parametrization of the arc.
TL;DR: This paper first explores the use of Sturm sequences to efficiently compute histograms of eigenvalues for symmetric tridiagonal matrices and applies these ideas to random matrix ensembles such as the β-Hermite ensemble.
Abstract: This paper proposes that the study of Sturm sequences is invaluable in the numerical computation and theoretical derivation of eigenvalue distributions of random matrix ensembles. We first explore the use of Sturm sequences to efficiently compute histograms of eigenvalues for symmetric tridiagonal matrices and apply these ideas to random matrix ensembles such as the β-Hermite ensemble. Using our techniques, we reduce the time to compute a histogram of the eigenvalues of such a matrix from O(n 2+m) to O(mn) time where n is the dimension of the matrix and m is the number of bins (with arbitrary bin centers and widths) desired in the histogram (m is usually much smaller than n). Second, we derive analytic formulas in terms of iterated multivariate integrals for the eigenvalue distribution and the largest eigenvalue distribution for arbitrary symmetric tridiagonal random matrix models. As an example of the utility of this approach, we give a derivation of both distributions for the β-Hermite random matrix ensemble (for general β). Third, we explore the relationship between the Sturm sequence of a random matrix and its shooting eigenvectors. We show using Sturm sequences that assuming the eigenvector contains no zeros, the number of sign changes in a shooting eigenvector of parameter λ is equal to the number of eigenvalues greater than λ. Finally, we use the techniques presented in the first section to experimentally demonstrate a O(log n) growth relationship between the variance of histogram bin values and the order of the β-Hermite matrix ensemble.
TL;DR: This paper seeks to bridge the two major algorithmic approaches to sparse signal recovery from an incomplete set of linear measurements--L1-minimization methods and iterative methods (Matching Pursuit and Matching Pursuit).
Abstract: This paper seeks to bridge the two major algorithmic approaches to sparse signal recovery from an incomplete set of linear measurements--L1-minimization methods and iterative methods (Matching Purs...
TL;DR: Introducing the very general class of uniformly absolutely continuous probability models for the random matrix A, it is shown that the distribution tails of C(A) decrease at algebraic rates, both for the Goffin–Cheung–Cucker number CG and the Renegar number CR.
Abstract: We consider the conic feasibility problem associated with the linear homogeneous system Ax≤0, x≠0 The complexity of iterative algorithms for solving this problem depends on a condition number C(A) When studying the typical behavior of algorithms under stochastic input, one is therefore naturally led to investigate the fatness of the tails of the distribution of C(A) Introducing the very general class of uniformly absolutely continuous probability models for the random matrix A, we show that the distribution tails of C(A) decrease at algebraic rates, both for the Goffin–Cheung–Cucker number C G and the Renegar number C R The exponent that drives the decay arises naturally in the theory of uniform absolute continuity, which we also develop in this paper In the case of C G , we also discuss lower bounds on the tail probabilities and show that there exist absolutely continuous input models for which the tail decay is subalgebraic
TL;DR: Generalized tractability is obtained for T(x,y), which is a tractability function which is non-decreasing in both variables and grows slower than exponentially to infinity and which is known that T must go to infinity faster than polynomially.
Abstract: We continue the study of generalized tractability initiated in our previous paper “Generalized tractability for multivariate problems, Part I: Linear tensor product problems and linear information”, J. Complex. 23:262–295, 2007. We study linear tensor product problems for which we can compute linear information which is given by arbitrary continuous linear functionals. We want to approximate an operator S d given as the d-fold tensor product of a compact linear operator S 1 for d=1,2,…, with ‖S 1‖=1 and S 1 having at least two positive singular values.
Let n(e,S d ) be the minimal number of information evaluations needed to approximate S d to within e∈[0,1]. We study generalized tractability by verifying when n(e,S d ) can be bounded by a multiple of a power of T(e −1,d) for all (e −1,d)∈Ω⊆[1,∞)×ℕ. Here, T is a tractability function which is non-decreasing in both variables and grows slower than exponentially to infinity. We study the exponent of tractability which is the smallest power of T(e −1,d) whose multiple bounds n(e,S d ). We also study weak tractability, i.e., when $\lim_{\varepsilon^{-1}+d\to\infty,(\varepsilon^{-1},d)\in \varOmega}\ln n(\varepsilon,S_{d})/(\varepsilon^{-1}+d)=0$.
In our previous paper, we studied generalized tractability for proper subsets Ω of [1,∞)×ℕ, whereas in this paper we take the unrestricted domain Ω unr=[1,∞)×ℕ.
We consider the three cases for which we have only finitely many positive singular values of S 1, or they decay exponentially or polynomially fast. Weak tractability holds for these three cases, and for all linear tensor product problems for which the singular values of S 1 decay slightly faster than logarithmically. We provide necessary and sufficient conditions on the function T such that generalized tractability holds. These conditions are obtained in terms of the singular values of S 1 and mostly asymptotic properties of T. The tractability conditions tell us how fast T must go to infinity. It is known that T must go to infinity faster than polynomially. We show that generalized tractability is obtained for T(x,y)=x 1+ln y . We also study tractability functions T of product form, T(x,y)=f 1(x)f 2(x). Assume that a i =lim inf x→∞(ln ln f i (x))/(ln ln x) is finite for i=1,2. Then generalized tractability takes place iff $$a_{i}>1\quad\mbox{and}\quad(a_{1}-1)(a_{2}-1)\ge1$$, and if (a 1−1)(a 2−1)=1 then we need to assume one more condition given in the paper. If (a 1−1)(a 2−1)>1 then the exponent of tractability is zero, and if (a 1−1)(a 2−1)=1 then the exponent of tractability is finite. It is interesting to add that for T being of the product form, the tractability conditions as well as the exponent of tractability depend only on the second singular eigenvalue of S 1 and they do not depend on the rate of their decay.
Finally, we compare the results obtained in this paper for the unrestricted domain Ω unr with the results from our previous paper obtained for the restricted domain Ω res=[1,∞)×{1,2,…,d *}∪[1,e 0−1)×ℕ with d *≥1 and e 0∈(0,1). In general, the tractability results are quite different. We may have generalized tractability for the restricted domain and no generalized tractability for the unrestricted domain which is the case, for instance, for polynomial tractability T(x,y)=xy. We may also have generalized tractability for both domains with different or with the same exponents of tractability.
TL;DR: A new coding scheme for general real-valued Lévy processes is introduced and its performance with respect to Lp[0,1]-norm distortion under different complexity constraints is controlled.
Abstract: We introduce a new coding scheme for general real-valued Levy processes and control its performance with respect to L p [0,1]-norm distortion under different complexity constraints. We also establish lower bounds that prove the optimality of our coding scheme in many cases.
TL;DR: This paper deepen the theoretical study of the geometric structure of a balanced complex polytope (b.c.p.), which is the generalization of a real centrally symmetric polytopes to the complex space, and proposes a constructive algorithm for the representation of its facets in terms of their associated linear functionals.
Abstract: In this paper, we deepen the theoretical study of the geometric structure of a balanced complex polytope (b.c.p.), which is the generalization of a real centrally symmetric polytope to the complex space. We also propose a constructive algorithm for the representation of its facets in terms of their associated linear functionals. The b.c.p.s are used, for example, as a tool for the computation of the joint spectral radius of families of matrices. For the representation of real polytopes, there exist well-known algorithms such as, for example, the Beneath–Beyond method. Our purpose is to modify and adapt this method to the complex case by exploiting the geometric features of the b.c.p. However, due to the significant increase in the difficulty of the problem when passing from the real to the complex case, in this paper, we confine ourselves to examine the two-dimensional case. We also propose an algorithm for the computation of the norm the unit ball of which is a b.c.p.
TL;DR: In this article, the authors classify two-component evolution equations, with homogeneous diagonal linear part, admitting infinitely many approximate symmetries, including the Skolem-Mahler-Lech theorem, an algorithm of Smyth, and results on diophantine equations in roots of unity obtained by Beukers.
Abstract: We globally classify two-component evolution equations, with homogeneous diagonal linear part, admitting infinitely many approximate symmetries. Important ingredients are the symbolic calculus of Gel’fand and Dikiĭ, the Skolem–Mahler–Lech theorem, an algorithm of Smyth, and results on diophantine equations in roots of unity obtained by Beukers.
TL;DR: The main result establishes the word problem to be computationally equivalent to the Halting Problem for such machines, giving the first non-trivial example of a problem complete, that is, computationally universal for this model.
Abstract: The word problem for discrete groups is well known to be undecidable by a Turing Machine; more precisely, it is reducible both to and from and thus equivalent to the discrete Halting Problem. The present work introduces and studies a real extension of the word problem for a certain class of groups which are presented as quotient groups of a free group and a normal subgroup. As a main difference to discrete groups these groups may be generated by uncountably many generators with index running over certain sets of real numbers. We study the word problem for such groups within the Blum–Shub–Smale (BSS) model of real number computation. The main result establishes the word problem to be computationally equivalent to the Halting Problem for such machines. It thus gives the first non-trivial example of a problem complete, that is, computationally universal for this model.
TL;DR: Persistent homology has proven to be a useful tool in a variety of contexts, including the recognition and measurement of shape characteristics of surfaces in ℝ3. as mentioned in this paper.
Abstract: Persistent homology has proven to be a useful tool in a variety of contexts, including the recognition and measurement of shape characteristics of surfaces in ℝ3. Persistence pairs homology classes...