scispace - formally typeset
Search or ask a question
Journal ArticleDOI

On Local Convergence of the Method of Alternating Projections

01 Apr 2016-Foundations of Computational Mathematics (Springer US)-Vol. 16, Iss: 2, pp 425-455
TL;DR: In this article, the authors proved local convergence of alternating projections between subanalytic sets under a mild regularity hypothesis on one of the sets, and showed that the speed of convergence is O(k √ √ σ(k − σ ) for some constant σ √ n, σ (n) for some σ σ = (0, √ N) √ (n − ρ) for any σ > 0.
Abstract: The method of alternating projections is a classical tool to solve feasibility problems. Here we prove local convergence of alternating projections between subanalytic sets $$A,B$$A,B under a mild regularity hypothesis on one of the sets. We show that the speed of convergence is $${\mathcal {O}}(k^{-\rho })$$O(k-?) for some $$\rho \in (0,\infty )$$??(0,?).

Content maybe subject to copyright    Report

Citations
More filters
Journal ArticleDOI
TL;DR: This work considers the method of alternating projections for finding a point in the intersection of two closed sets and proves local linear convergence and subsequence convergence when the two sets are semi-algebraic and bounded, but not necessarily transversal.
Abstract: We consider the method of alternating projections for finding a point in the intersection of two closed sets, possibly nonconvex. Assuming only the standard transversality condition (or a weaker version thereof), we prove local linear convergence. When the two sets are semi-algebraic and bounded, but not necessarily transversal, we nonetheless prove subsequence convergence.

93 citations

Journal ArticleDOI
TL;DR: In this paper, the authors show that with a suitable initialization procedure, the classical alternating projections (Gerchberg-Saxton) succeeds with high probability when $m\geq Cn$, for some $C>0$.
Abstract: We consider a phase retrieval problem, where we want to reconstruct a $n$ -dimensional vector from its phaseless scalar products with $m$ sensing vectors, independently sampled from complex normal distributions. We show that, with a suitable initialization procedure, the classical algorithm of alternating projections (Gerchberg–Saxton) succeeds with high probability when $m\geq Cn$ , for some $C>0$ . We conjecture that this result is still true when no special initialization procedure is used, and present numerical experiments that support this conjecture.

76 citations

Posted Content
TL;DR: It is conjecture that the classical algorithm of alternating projections (Gerchberg–Saxton) succeeds with high probability when no special initialization procedure is used, and it is conjectured that this result is still true when nospecial initialization process is used.
Abstract: We consider a phase retrieval problem, where we want to reconstruct a $n$-dimensional vector from its phaseless scalar products with $m$ sensing vectors. We assume the sensing vectors to be independently sampled from complex normal distributions. We propose to solve this problem with the classical non-convex method of alternating projections. We show that, when $m\geq Cn$ for $C$ large enough, alternating projections succeed with high probability, provided that they are carefully initialized. We also show that there is a regime in which the stagnation points of the alternating projections method disappear, and the initialization procedure becomes useless. However, in this regime, $m$ has to be of the order of $n^2$. Finally, we conjecture from our numerical experiments that, in the regime $m=O(n)$, there are stagnation points, but the size of their attraction basin is small if $m/n$ is large enough, so alternating projections can succeed with probability close to $1$ even with no special initialization.

72 citations

Journal ArticleDOI
TL;DR: In this article, the authors synthesize and unify notions of regularity, both of individual sets and of collections of sets, as they appear in the convergence theory of projection methods for consistent feasibility problems.
Abstract: We synthesize and unify notions of regularity, both of individual sets and of collections of sets, as they appear in the convergence theory of projection methods for consistent feasibility problems. Several new characterizations of regularities are presented which shed light on the relations between seemingly different ideas and point to possible necessary conditions for local linear convergence of fundamental algorithms.

57 citations

Journal ArticleDOI
TL;DR: In this paper, the authors develop a framework for quantitative convergence analysis of Picard iterations of expansive set-valued fixed point mappings, and prove local linear convergence of nonconvex cyclic projections for inconsistent (and consistent) feasibility problems.
Abstract: We develop a framework for quantitative convergence analysis of Picard iterations of expansive set-valued fixed point mappings. There are two key components of the analysis. The first is a natural generalization of single-valued averaged mappings to expansive set-valued mappings that characterizes a type of strong calmness of the fixed point mapping. The second component to this analysis is an extension of the well-established notion of metric subregularity - or inverse calmness - of the mapping at fixed points. Convergence of expansive fixed point iterations is proved using these two properties, and quantitative estimates are a natural by-product of the framework. To demonstrate the application of the theory, we prove, for the first time, a number of results showing local linear convergence of nonconvex cyclic projections for inconsistent (and consistent) feasibility problems, local linear convergence of the forward-backward algorithm for structured optimization without convexity, strong or otherwise, and local linear convergence of the Douglas-Rachford algorithm for structured nonconvex minimization. This theory includes earlier approaches for known results, convex and nonconvex, as special cases.

50 citations

References
More filters
Book
01 Jan 1997
TL;DR: The geometry of subanalytic and semialgebraic sets as mentioned in this paper is a perfect book that comes from a great author to share with you, it offers the best experience and lesson to take, not only take, but also learn.
Abstract: geometry of subanalytic and semialgebraic sets. Book lovers, when you need a new book to read, find the book here. Never worry not to find what you need. Is the geometry of subanalytic and semialgebraic sets your needed book now? That's true; you are really a good reader. This is a perfect book that comes from great author to share with you. The book offers the best experience and lesson to take, not only take, but also learn.

209 citations

Journal ArticleDOI
TL;DR: A survey of Fréchet subdifferentiation can be found in this article, where the authors discuss fuzzy results in terms of simple subdifferentials calculated at some points arbitrarily close to the point under consideration.
Abstract: This survey is devoted to some aspects of the theory of Fréchet subdifferentiation. The selection of the material reflects the interests of the author and is far from being complete. The paper contains definitions and statements of some important results in the field with very few proofs. The author hopes that reading the paper will not be difficult even for those mathematicians whose main scientific interests are not in the field of nonsmooth analysis. The variety of different subdifferentials known by now can be divided into two large groups: “simple” subdifferentials and “strict” subdifferentials. A simple subdifferential is defined at a given point and it does not take into account “differential” properties of a function in its neighborhood. Usually, such subdifferentials generalize some classical differentiability notions (Fréchet, Gâteaux, Dini, etc.). They are not widely used directly because of rather poor calculus. Contrary to simple subdifferentials, the definitions of strict subdifferentials incorporate differential properties of a function near a given point. Usually, strict subdifferentials can be represented as (some kinds of) limits of simple ones. This procedure makes them generalizations of the notion of a strict derivative [14], enriches their properties, and allows constructing satisfactory calculus. The examples of limiting subdifferentials are the generalized differential (the limiting Fréchet subdifferential) [49, 53, 63, 66, 67] and the approximate subdifferential (the limiting Dini subdifferential) [38, 39, 41, 43]. The famous generalized gradient of Clarke [15,17] can also be considered as being a strict subdifferential. The Warga’s derivate container [98,101] also belongs to this class. The limiting subdifferentials proved to be very efficient in nonsmooth analysis and optimization (see [17, 38, 41–44, 49, 50, 52, 53, 62, 63, 67, 69, 74, 75, 93, 95, 97–102]), especially in finite dimensions. When applying limiting subdifferentials in infinite-dimensional spaces, one must be careful about nontriviality of limits in the weak∗ topology. Additional regularity conditions are needed (compact epi-Lipschitzness [7], sequential normal compactness, partial sequential normal compactness [74,75,77], etc.). On the other hand, it is possible to formulate the results without taking limits and thus avoid the above-mentioned difficulties. Such statements are formulated (without additional regularity conditions) in terms of simple subdifferentials calculated at some points arbitrarily close to the point under consideration. They are usually referred to as fuzzy results [10, 12, 27–29, 40, 45, 62, 76, 78, 104]. In general, such results are stronger than the corresponding statements in terms of limiting subdifferentials. In this paper, we discuss only fuzzy results. The paper consists of three sections. Section 1 is devoted to definitions and elementary properties of Fréchet subdifferentials, normal cones, and coderivatives. It partly follows the earlier papers [48, 51], some parts of which have never been published. The main fuzzy results (from the author’s standpoint) in terms of Fréchet subdifferentials are presented in Sec. 2. Some of them are formulated by using strict δ-subdifferentials [55, 61]. The extended extremality notions [61] are discussed in Sec. 3. Being weaker than the traditional definitions, they describe some “almost extremal” points for which the known dual necessary conditions in terms of Fréchet subdifferentials become sufficient. Adopting these extended extremality notions leads to a form of duality in nonsmooth nonconvex optimization. Some constants are defined in the paper which simplify the definitions and statements of the results.

183 citations

Journal ArticleDOI
TL;DR: In this article, a notion of local subfirm nonexpansiveness with respect to the intersection is introduced for consistent feasibility problems, together with a coercivity condition that relates to the regularity of the collection of sets at points in the intersection, yields local linear convergence of AP for a wide class of nonconvex problems.
Abstract: We consider projection algorithms for solving (nonconvex) feasibility problems in Euclidean spaces. Of special interest are the method of alternating projections (AP) and the Douglas--Rachford algorithm (DR). In the case of convex feasibility, firm nonexpansiveness of projection mappings is a global property that yields global convergence of AP and for consistent problems DR. A notion of local subfirm nonexpansiveness with respect to the intersection is introduced for consistent feasibility problems. This, together with a coercivity condition that relates to the regularity of the collection of sets at points in the intersection, yields local linear convergence of AP for a wide class of nonconvex problems and even local linear convergence of nonconvex instances of the DR algorithm.

180 citations

Journal ArticleDOI
TL;DR: In this article, the authors generalize the MOSP to collections of approximately compact sets in metric spaces and define a sequence of successive projections (SOSP) in such a context and then proceed to establish conditions for the convergence of a SOSP to a solution point.
Abstract: Many problems in applied mathematics can be abstracted into finding a common point of a finite collection of sets. If all the sets are closed and convex in a Hilbert space, the method of successive projections (MOSP) has been shown to converge to a solution point, i.e., a point in the intersection of the sets. These assumptions are however not suitable for a broad class of problems. In this paper, we generalize the MOSP to collections of approximately compact sets in metric spaces. We first define a sequence of successive projections (SOSP) in such a context and then proceed to establish conditions for the convergence of a SOSP to a solution point. Finally, we demonstrate an application of the method to digital signal restoration.

125 citations

Journal ArticleDOI
Veit Elser1
TL;DR: An algorithm for determining crystal structures from diffraction data is described which does not rely on the usual reciprocal-space formulations of atomicity, and implements atomicity constraints in real space as well as intensity constraints in reciprocal space by projections that restore each constraint with the minimal modification of the scattering density.
Abstract: An algorithm for determining crystal structures from diffraction data is described which does not rely on the usual reciprocal-space formulations of atomicity. The new algorithm implements atomicity constraints in real space, as well as intensity constraints in reciprocal space, by projections that restore each constraint with the minimal modification of the scattering density. To recover the true density, the two projections are combined into a single operation, the difference map, which is iterated until the magnitude of the density modification becomes acceptably small. The resulting density, when acted upon by a single additional operation, is by construction a density that satisfies both intensity and atomicity constraints. Numerical experiments have yielded solutions for atomic resolution X-ray data sets with over 400 non-hydrogen atoms, as well as for neutron data, where positivity of the density cannot be invoked.

118 citations