Thank you very much for reading perturbation analysis of optimization problems. Maybe you have knowledge that, people have look hundreds times for their favorite books like this perturbation analysis of optimization problems, but end up in malicious downloads. Rather than enjoying a good book with a cup of tea in the afternoon, instead they are facing with some malicious virus inside their computer. perturbation analysis of optimization problems is available in our book collection an online access to it is set as public so you can get it instantly. Our books collection saves in multiple countries, allowing you to get the most less latency time to download any of our books like this one. Merely said, the perturbation analysis of optimization problems is universally compatible with any devices to read.

Perturbation Analysis Of Optimization Problems

Operator-splitting schemes are iterative algorithms for solving many types of numerical problems. A lot is known about these methods: they converge, and in many cases we know how quickly they converge. But when they are applied to optimization problems, there is a gap in our understanding: The theoretical speed of operator-splitting schemes is nearly always measured in the ergodic sense, but ergodic operator-splitting schemes are rarely used in practice. In this chapter, we tackle the discrepancy between theory and practice and uncover fundamental limits of a class of operator-splitting schemes. Our surprising conclusion is that the relaxed Peaceman-Rachford splitting algorithm, a version of the Alternating Direction Method of Multipliers (ADMM), is nearly as fast as the proximal point algorithm in the ergodic sense and nearly as slow as the subgradient method in the nonergodic sense. A large class of operator-splitting schemes extend from the relaxed Peaceman-Rachford splitting algorithm. Our results show that this class of operator-splitting schemes is also nearly as slow as the subgradient method. The tools we create in this chapter can also be used to prove nonergodic convergence rates of more general splitting schemes, so they are interesting in their own right.

http://www.optimization-online.org/DB_FILE/2014/06/4394.pdf

Convergence Rate Analysis of Several Splitting Schemes

This paper shows that error bounds can be used as effective tools for deriving complexity results for first-order descent methods in convex minimization. In a first stage, this objective led us to revisit the interplay between error bounds and the Kurdyka-Łojasiewicz (KL) inequality. One can show the equivalence between the two concepts for convex functions having a moderately flat profile near the set of minimizers (as those of functions with Holderian growth). A counterexample shows that the equivalence is no longer true for extremely flat functions. This fact reveals the relevance of an approach based on KL inequality. In a second stage, we show how KL inequalities can in turn be employed to compute new complexity bounds for a wealth of descent methods for convex problems. Our approach is completely original and makes use of a one-dimensional worst-case proximal sequence in the spirit of the famous majorant method of Kantorovich. Our result applies to a very simple abstract scheme that covers a wide class of descent methods. As a byproduct of our study, we also provide new results for the globalization of KL inequalities in the convex framework. Our main results inaugurate a simple method: derive an error bound, compute the desingularizing function whenever possible, identify essential constants in the descent method and finally compute the complexity using the one-dimensional worst case proximal sequence. Our method is illustrated through projection methods for feasibility problems, and through the famous iterative shrinkage thresholding algorithm (ISTA), for which we show that the complexity bound is of the form \(O(q^{k})\) where the constituents of the bound only depend on error bound constants obtained for an arbitrary least squares objective with \(\ell ^1\) regularization.

/pdf/from-error-bounds-to-the-complexity-of-first-order-descent-27va13vg7e.pdf

From error bounds to the complexity of first-order descent methods for convex functions

The proximal gradient algorithm for minimizing the sum of a smooth and nonsmooth convex function often converges linearly even without strong convexity. One common reason is that a multiple of the step length at each iteration may linearly bound the “error”—the distance to the solution set. We explain the observed linear convergence intuitively by proving the equivalence of such an error bound to a natural quadratic growth condition. Our approach generalizes to linear and quadratic convergence analysis for proximal methods (of Gauss-Newton type) for minimizing compositions of nonsmooth functions with smooth mappings. We observe incidentally that short step-lengths in the algorithm indicate near-stationarity, suggesting a reliable termination criterion.

http://www.optimization-online.org/DB_FILE/2016/02/5337.pdf

Error Bounds, Quadratic Growth, and Linear Convergence of Proximal Methods

This work proposes block-coordinate fixed point algorithms with applications to nonlinear analysis and optimization in Hilbert spaces. The asymptotic analysis relies on a notion of stochastic quasi-Fejer monotonicity, which is thoroughly investigated. The iterative methods under consideration feature random sweeping rules to select arbitrarily the blocks of variables that are activated over the course of the iterations and they allow for stochastic errors in the evaluation of the operators. Algorithms using quasi-nonexpansive operators or compositions of averaged nonexpansive operators are constructed, and weak and strong convergence results are established for the sequences they generate. As a by-product, novel block-coordinate operator splitting methods are obtained for solving structured monotone inclusion and convex minimization problems. In particular, the proposed framework leads to random block-coordinate versions of the Douglas--Rachford and forward-backward algorithms and of some of their variant...

/pdf/stochastic-quasi-fejer-block-coordinate-fixed-point-3h1c6i911f.pdf

Stochastic Quasi-Fejér Block-Coordinate Fixed Point Iterations with Random Sweeping

The paper concerns the study of new classes of parametric optimization problems of the so-called infinite programming that are generally defined on infinite-dimensional spaces of decision variables and contain, among other constraints, infinitely many inequality constraints. These problems reduce to semi-infinite programs in the case of finite-dimensional spaces of decision variables. We focus on DC infinite programs with objectives given as the difference of convex functions subject to convex inequality constraints. The main results establish efficient upper estimates of certain subdifferentials of (intrinsically nonsmooth) value functions in DC infinite programs based on advanced tools of variational analysis and generalized differentiation. The value/marginal functions and their subdifferential estimates play a crucial role in many aspects of parametric optimization including well-posedness and sensitivity. In this paper we apply the obtained subdifferential estimates to establishing verifiable conditions for the local Lipschitz continuity of the value functions and deriving necessary optimality conditions in parametric DC infinite programs and their remarkable specifications. Finally, we employ the value function approach and the established subdifferential estimates to the study of bilevel finite and infinite programs with convex data on both lower and upper level of hierarchical optimization. The results obtained in the paper are new not only for the classes of infinite programs under consideration but also for their semi-infinite counterparts.

Subdifferentials of value functions and optimality conditions for DC and bilevel infinite and semi-infinite programs

This paper concerns a closedness condition called (CC) involving a convex function and a convex constrained system. This type of condition has played an important role in the study of convex optimization problems. Our aim is to establish several characterizations of this condition and to apply them to study problems of minimizing a DC function under a cone-convex constraint and a set constraint. First, we establish several so-called ‘Toland–Fenchel–Lagrange’ duality theorems. As consequences, various versions of generalized Farkas lemmas in dual forms for systems involving convex and DC functions are derived. Then, we establish optimality conditions for DC problem under convex constraints. Optimality conditions for convex problems and problems of maximizing a convex function under convex constraints are given as well. Most of the results are established under the (CC) condition. This article serves as a link between several corresponding known ones published recently for DC programs and for convex programs.

/pdf/a-closedness-condition-and-its-applications-to-dc-programs-3tzypen61r.pdf

A closedness condition and its applications to DC programs with convex constraints

This paper sheds new light on several interrelated topics of second-order variational analysis, both in finite and infinite-dimensional settings. We establish new relationships between second-order growth conditions on functions, the basic properties of metric regularity and subregularity of the limiting subdifferential, tilt-stability of local minimizers, and positive-definiteness/semidefiniteness properties of the second-order subdifferential (or generalized Hessian).

Second-order growth, tilt stability, and metric regularity of the subdifferential

The paper concerns a systematic study of full stability in general optimization models including its conventional Lipschitzian version as well as the new Holderian one. We derive various characteri...

Full Lipschitzian and Hölderian Stability in Optimization with Applications to Mathematical Programming and Optimal Control

In this paper we focus on the convergence analysis of the forward–backward splitting method for solving nonsmooth optimization problems in Hilbert spaces when the objective function is the sum of two convex functions. Assuming that one of the functions is Frechet differentiable and using two new linesearches, the weak convergence is established without any Lipschitz continuity assumption on the gradient. Furthermore, we obtain many complexity results of cost values at the iterates when the stepsizes are bounded below by a positive constant.

Tran T. A. Nghia

Papers

Subdifferentials of value functions and optimality conditions for DC and bilevel infinite and semi-infinite programs

A closedness condition and its applications to DC programs with convex constraints

Second-order growth, tilt stability, and metric regularity of the subdifferential

Full Lipschitzian and Hölderian Stability in Optimization with Applications to Mathematical Programming and Optimal Control

On the convergence of the forward–backward splitting method with linesearches