Showing papers in "Optimization in 2015"
TL;DR: In this paper, a scaled vector transport is introduced to improve the conjugate gradient method so that the generated sequences may have a global convergence property under a relaxed assumption, and the proposed algorithm is theoretically proved and numerically observed with examples.
Abstract: This article deals with the conjugate gradient method on a Riemannian manifold with interest in global convergence analysis. The existing conjugate gradient algorithms on a manifold endowed with a vector transport need the assumption that the vector transport does not increase the norm of tangent vectors, in order to confirm that generated sequences have a global convergence property. In this article, the notion of a scaled vector transport is introduced to improve the algorithm so that the generated sequences may have a global convergence property under a relaxed assumption. In the proposed algorithm, the transported vector is rescaled in case its norm has increased during the transport. The global convergence is theoretically proved and numerically observed with examples. In fact, numerical experiments show that there exist minimization problems for which the existing algorithm generates divergent sequences, but the proposed algorithm generates convergent sequences.
106 citations
TL;DR: In this paper, necessary and sufficient criteria for metric subregularity of set-valued mappings between general metric or Banach spaces are treated in the framework of the theory of error bounds for a special family of extended real-valued functions of two variables.
Abstract: Necessary and sufficient criteria for metric subregularity (or calmness) of set-valued mappings between general metric or Banach spaces are treated in the framework of the theory of error bounds for a special family of extended real-valued functions of two variables. A classification scheme for the general error bound and metric subregularity criteria is presented. The criteria are formulated in terms of several kinds of primal and subdifferential slopes.
100 citations
TL;DR: In this paper, weakly convergent algorithms for finding a zero of the sum of a maximally monotone operator, a cocoercive operator, and the normal cone to a closed vector subspace of a real Hilbert space are presented.
Abstract: We provide two weakly convergent algorithms for finding a zero of the sum of a maximally monotone operator, a cocoercive operator, and the normal cone to a closed vector subspace of a real Hilbert space. The methods exploit the intrinsic structure of the problem by activating explicitly the cocoercive operator in the first step, and taking advantage of a vector space decomposition in the second step. The second step of the first method is a Douglas–Rachford iteration involving the maximally monotone operator and the normal cone. In the second method, it is a proximal step involving the partial inverse of the maximally monotone operator with respect to the vector subspace. Connections between the proposed methods and other methods in the literature are provided. Applications to monotone inclusions with finitely many maximally monotone operators and optimization problems are examined.
87 citations
TL;DR: In this article, an iterative process converges strongly to a common minimum-norm solution of a variational inequality problem for an -inverse strongly monotone mapping and a fixed point of relatively non-expansive mapping in Banach spaces.
Abstract: We introduce an iterative process which converges strongly to a common minimum-norm solution of a variational inequality problem for an -inverse strongly monotone mapping and a fixed point of relatively non-expansive mapping in Banach spaces. Our theorems improve and unify most of the results that have been proved for this important class of non-linear operators.
74 citations
TL;DR: In this paper, a general class of quasi-none-expansive operators with nonempty fixed-point sets is introduced for the split common fixed point problem, and a simultaneous iterative algorithm with weak convergence is presented.
Abstract: Let , and be real Hilbert spaces, let and be two bounded linear operators Moudafi introduced simultaneous iterative algorithms with weak convergence for the following split common fixed-point problem:SectionDisplay where and are two firmly quasi-nonexpansive operators with nonempty fixed-point sets and Note that, by taking and , we recover the split common fixed-point problem originally introduced by Cesnor and Segal However, to employ Moudafi’s algorithms, one needs to know a prior norm (or at least an estimate of the norm) of the bounded linear operators To estimate the norm of an operator is very difficult, if it is not an impossible task In this paper, we will continue to consider the split common fixed-point problem (1) governed by the general class of quasi-nonexpansive operators We introduce a simultaneous iterative algorithm with a way of selecting the stepsizes such that the implementation of the algorithm does not need any prior information about the operator norms The weak convergence
67 citations
TL;DR: In this paper, new numerical algorithms are introduced for finding the solution of a variational inequality problem whose constraint set is the common elements of the set of fixed points of a demicontractive mapping and the sets of solutions of an equilibrium problem for a monotone mapping in a real Hilbert space.
Abstract: In this paper, new numerical algorithms are introduced for finding the solution of a variational inequality problem whose constraint set is the common elements of the set of fixed points of a demicontractive mapping and the set of solutions of an equilibrium problem for a monotone mapping in a real Hilbert space. The strong convergence of the iterates generated by these algorithms is obtained by combining a viscosity approximation method with an extragradient method. First, this is done when the basic iteration comes directly from the extragradient method, under a Lipschitz-type condition on the equilibrium function. Then, it is shown that this rather strong condition can be omitted when an Armijo-backtracking linesearch is incorporated into the extragradient iteration. The particular case of variational inequality problems is also examined.
64 citations
TL;DR: In this article, the proximal point method for finding minima of a special class of nonconvex functions on a Hadamard manifold is presented, and it is proved that each accumulation point of this sequence satisfies the necessary optimality conditions.
Abstract: In this article, we present the proximal point method for finding minima of a special class of nonconvex function on a Hadamard manifold. The well definedness of the sequence generated by the proximal point method is established. Moreover, it is proved that each accumulation point of this sequence satisfies the necessary optimality conditions and, under additional assumptions, its convergence for a minima is obtained.
62 citations
TL;DR: This annotated bibliography includes books and review papers on, or related to, projection methods that the authors know about, use and like.
Abstract: Projections onto sets are used in a wide variety of methods in optimization theory but not every method that uses projections really belongs to the class of projection methods as we mean it here. Here, projection methods are iterative algorithms that use projections onto sets while relying on the general principle that when a family of (usually closed and convex) sets is present, then projections (or approximate projections) onto the given individual sets are easier to perform than projections onto other sets (intersections, image sets under some transformation, etc.) that are derived from the given family of individual sets. Projection methods employ projections (or approximate projections) onto convex sets in various ways. They may use different kinds of projections and, sometimes, even use different projections within the same algorithm. They serve to solve a variety of problems which are either of the feasibility or the optimization types. They have different algorithmic structures, of which some are ...
56 citations
TL;DR: In this article, a projection algorithm with a way of selecting the stepsizes such that the implementation of the projection algorithm does not need any priori information about the operator norms is presented.
Abstract: The split equality problem has extraordinary utility and broad applicability in many areas of applied mathematics. Recently, Moudafi proposed an alternating CQ algorithm and its relaxed variant to solve it. However, to employ Moudafi’s algorithms, one needs to know a priori norm (or at least an estimate of the norm) of the bounded linear operators (matrices in the finite-dimensional framework). To estimate the norm of an operator is very difficult, but not an impossible task. It is the purpose of this paper to introduce a projection algorithm with a way of selecting the stepsizes such that the implementation of the algorithm does not need any priori information about the operator norms. We also practise this way of selecting stepsizes for variants of the projection algorithm, including a relaxed projection algorithm where the two closed convex sets are both level sets of convex functions, and a viscosity algorithm. Both weak and strong convergence are investigated.
53 citations
TL;DR: In this paper, a multidimensional optimization problem in the tropical mathematics setting is examined. But the problem involves the minimization of a non-linear function defined on a finite-dimensional semimodule over an idempotent semiield subject to linear inequality constraints.
Abstract: We examine a multidimensional optimization problem in the tropical mathematics setting. The problem involves the minimization of a non-linear function defined on a finite-dimensional semimodule over an idempotent semifield subject to linear inequality constraints. We start with an overview of known tropical optimization problems with linear and non-linear objective functions. A short introduction to tropical algebra is provided to offer a formal framework for solving the problem under study. As a preliminary result, a solution to a linear inequality with an arbitrary matrix is presented. We describe an example optimization problem drawn from project scheduling and then offer a general representation of the problem. To solve the problem, we introduce an additional variable and reduce the problem to the solving of a linear inequality, in which the variable plays the role of a parameter. A necessary and sufficient condition for the inequality to hold is used to evaluate the parameter, whereas the solution to...
51 citations
TL;DR: In this paper, continuous and discrete dynamical systems which aim at solving inclusions governed by structured monotone operators, where is the subdifferential of a convex lower semicontinuous function, and is a monotonic cocoercive operator, are introduced.
Abstract: In a Hilbert framework, we introduce continuous and discrete dynamical systems which aim at solving inclusions governed by structured monotone operators , where is the subdifferential of a convex lower semicontinuous function , and is a monotone cocoercive operator. We first consider the extension to this setting of the regularized Newton dynamic with two potentials which was considered in Abbas, Attouch, Svaiter JOTA, 2014. Then, we revisit some related dynamical systems, namely the semigroup of contractions generated by , and the continuous gradient projection dynamic. By a Lyapunov analysis, we show the convergence properties of the orbits of these systems, thereby extending the known results. The time discretization of these dynamics gives various forward–backward splitting methods (some new) for solving structured monotone inclusions involving non-potential terms. The convergence of these algorithms is obtained under classical step size limitation. Perspectives are given in the field of numerical spl...
TL;DR: Censor et al. as discussed by the authors proposed a globally convergent algorithm for equilibrium problems with pseudomonotone bifunctions based on the idea of the subgradient extragradient method for solving variational inequalities.
Abstract: A globally convergent algorithm for equilibrium problems with pseudomonotone bifunctions is proposed. The algorithm is based on the idea of the subgradient extragradient method for solving variational inequalities proposed by Censor et al. [Y. Censor, A. Gibali, and S. Reich, The subgradient extragradient method for solving variational inequalities in Hilbert space, J. Optim. Theory Appl. 148 (2011), 318–335.] and Armijo linesearch techniques. In addition, we give a modified version of our algorithm for finding a common point of the solution set of equilibrium problems and the fixed point set of a nonexpansive mapping. We also analyse the weak convergence of both algorithms in a real Hilbert space.
TL;DR: In this paper, an ill-posed quasi-variational inequality with contaminated data can be stabilized by employing the elliptic regularization under suitable conditions, and conditions that ensure the boundedness of regularized solutions become sufficient solvability conditions.
Abstract: An ill-posed quasi-variational inequality with contaminated data can be stabilized by employing the elliptic regularization. Under suitable conditions, a sequence of bounded regularized solutions converges strongly to a solution of the original quasi-variational inequality. Moreover, the conditions that ensure the boundedness of regularized solutions, become sufficient solvability conditions. It turns out that the regularization theory is quite strong for quasi-variational inequalities with set-valued monotone maps but restrictive for generalized pseudo-monotone maps. The results are quite general and are applicable to ill-posed variational inequalities, hemi-variational inequalities, inverse problems, and split feasibility problem, among others.
TL;DR: In this article, two adaptive choices for the parameter of Dai-Liao conjugate gradient (CG) method are suggested, one of which is obtained by minimizing the distance between search directions of Dai -Liao method and a three-term CG method proposed by Zhang et al.
Abstract: Two adaptive choices for the parameter of Dai–Liao conjugate gradient (CG) method are suggested. One of which is obtained by minimizing the distance between search directions of Dai–Liao method and a three-term CG method proposed by Zhang et al. and the other one is obtained by minimizing Frobenius condition number of the search direction matrix. Global convergence analyses are made briefly. Numerical results are reported; they demonstrate effectiveness of the suggested adaptive choices.
TL;DR: In this article, the concepts of improvement set and -efficiency are introduced in a real locally convex Hausdorff topological vector space, and some properties of the improvement sets are given and a kind of proper efficiency, named as -Benson proper efficiency (BWE), which unifies some proper efficiency and approximate proper efficiency via the improvement set in vector optimization, is proposed.
Abstract: Starting from the innovative ideas of Chicco et al. (Vector optimization problems via improvement sets. J. Optim. Theory Appl. 2011;150:516–529), in this paper, the concepts of improvement set and -efficiency are introduced in a real locally convex Hausdorff topological vector space. Furthermore, some properties of the improvement sets are given and a kind of proper efficiency, named as -Benson proper efficiency, which unifies some proper efficiency and approximate proper efficiency, is proposed via the improvement sets in vector optimization. Moreover, the concept of -subconvexlikeness of set-valued maps is introduced via the improvement sets and an alternative theorem is proved. In the end, some scalarization theorems and Lagrange multiplier theorems of -Benson proper efficiency are established for a vector optimization problem with set-valued maps.
TL;DR: In this article, the existence of critical point and weak efficient point of vector optimization problem is studied and a sequence of points in n-dimension is generated using positive definite matrices like Quasi-Newton method.
Abstract: In this paper, existence of critical point and weak efficient point of vector optimization problem is studied. A sequence of points in n-dimension is generated using positive definite matrices like Quasi-Newton method. It is proved that accumulation points of this sequence are critical points or weak efficient points under different conditions. An algorithm is provided in this context. This method is free from any kind of priori chosen weighting factors or any other form of a priori ranking or ordering information for objective functions. Also, this method does not depend upon initial point. The algorithm is verified in numerical examples.
TL;DR: The convergence rate of the sequence of objective function values of a primal-dual proximal-point algorithm recently introduced in the literature for solving a primal convex optimization problem having as objective the sum of linearly composed infimal convolutions, nonsmooth and smooth convex functions and its Fenchel-type dual one is analyzed.
Abstract: In this paper, we analyse the convergence rate of the sequence of objective function values of a primal-dual proximal-point algorithm recently introduced in the literature for solving a primal convex optimization problem having as objective the sum of linearly composed infimal convolutions, nonsmooth and smooth convex functions and its Fenchel-type dual one. The theoretical part is illustrated by numerical experiments in image processing.
TL;DR: In decision-making problems where uncertainty plays a key role and decisions have to be taken prior to observing uncertainty, chance constraints are a strong modelling tool for defining safety of decisions.
Abstract: In decision-making problems where uncertainty plays a key role and decisions have to be taken prior to observing uncertainty, chance constraints are a strong modelling tool for defining safety of decisions. These constraints request that a random inequality system depending on a decision vector has to be satisfied with a high probability. The characteristics of the feasible set of such chance constraints depend on the constraint mapping of the random inequality system, the underlying law of uncertainty and the probability level. One characteristic of particular interest is convexity. Convexity can be shown under fairly general conditions on the underlying law of uncertainty and on the constraint mapping, regardless of the probability-level. In some situations, convexity can only be shown when the probability-level is high enough. This is defined as eventual convexity. In this paper, we will investigate further how eventual convexity can be assured for specially structured chance constraints involving Copu...
TL;DR: In this paper, the authors proposed an efficient global optimization method that transforms a 3D-ODRPP as a mixed-integer linear program using fewer extra 0-1 variables and constraints compared to existing deterministic approaches.
Abstract: The three-dimensional open dimension rectangular packing problem (3D-ODRPP) aims to pack a set of given rectangular boxes into a large rectangular container of minimal volume. This problem is an important issue in the shipping and moving industries. All the boxes can be any rectangular stackable objects with different sizes and may be freely rotated. The 3D-ODRPP is usually formulated as a mixed-integer non-linear programming problem. Most existing packing optimization methods cannot guarantee to find a globally optimal solution or are computationally inefficient. Therefore, this paper proposes an efficient global optimization method that transforms a 3D-ODRPP as a mixed-integer linear program using fewer extra 0–1 variables and constraints compared to existing deterministic approaches. The reformulated model can be solved to obtain a global optimum. Experimental results demonstrate the computational efficiency of the proposed approach in globally solving 3D-ODRPPs drawn from the literature and the practi...
TL;DR: In this paper, a relaxed self-adaptive CQ algorithm for solving the multiple sets split feasibility problem (MSFP) was proposed. But this algorithm has only weak convergence in the setting of infinite-dimensional Hilbert spaces.
Abstract: The multiple-sets split feasibility problem (MSFP) is to find a point belongs to the intersection of a family of closed convex sets in one space, such that its image under a linear transformation belongs to the intersection of another family of closed convex sets in the image space. Many iterative methods can be employed to solve the MSFP. Jinling Zhao et al. proposed a modification for the CQ algorithm and a relaxation scheme for this modification to solve the MSFP. The strong convergence of these algorithms are guaranteed in finite-dimensional Hilbert spaces. Recently Lopez et al. proposed a relaxed CQ algorithm for solving split feasibility problem, this algorithm can be implemented easily since it computes projections onto half-spaces and has no need to know a priori the norm of the bounded linear operator. However, this algorithm has only weak convergence in the setting of infinite-dimensional Hilbert spaces. In this paper, we introduce a new relaxed self-adaptive CQ algorithm for solving the MSFP wh...
TL;DR: The HGSO algorithm embeds predatory behaviour of artificial fish swarm algorithm (AFSA) into glowworm swarm optimization (GSO) algorithm and combines the GSO with differential evolution on the basis of a two-population co-evolution mechanism.
Abstract: In this paper, a novel hybrid glowworm swarm optimization (HGSO) algorithm is proposed. The HGSO algorithm embeds predatory behaviour of artificial fish swarm algorithm (AFSA) into glowworm swarm optimization (GSO) algorithm and combines the GSO with differential evolution on the basis of a two-population co-evolution mechanism. In addition, to overcome the premature convergence, the local search strategy based on simulated annealing is applied to make the search of GSO approach the true optimum solution gradually. Finally, several benchmark functions show that HGSO has faster convergence efficiency and higher computational precision, and is more effective for solving constrained multi-modal function optimization problems.
TL;DR: In this article, a projection-type method for variational inequalities from Euclidean spaces to Hadamard manifolds is proposed, which is well defined whether the solution set of the problem is non-empty or not, under weak assumptions.
Abstract: In this paper, we extend a projection-type method for variational inequalities from Euclidean spaces to Hadamard manifolds. The proposed method has the following nice features: (i) the algorithm is well defined whether the solution set of the problem is non-empty or not, under weak assumptions; (ii) if the solution set is non-empty, then the sequence generated by the method is convergent to the solution, which is closest to the initial point; and (iii) the existence of the solutions to variational inequalities can be verified through the behaviour of the generated sequence. The results presented in this paper generalize and improve some known results given in literatures.
TL;DR: The notion of quasi-relative interior was introduced by Borwein and Lewis in 1992 and applied for duality results in partially finite convex optimization problems as mentioned in this paper, and several articles were dedicated to duality result in infinite-dimensional scalar, vector and set-valued optimization problems using this notion.
Abstract: The notion of quasi-relative interior was introduced by Borwein and Lewis in 1992 and applied for duality results in partially finite convex optimization problems. In the last 10 years, several articles were dedicated to duality results in infinite-dimensional scalar, vector and set-valued optimization problems using this notion. The aim of this paper is to refine and discuss such results. We do this observing that the notion of quasi-relative interior is related to (non-proper) separation of a convex set and some of its elements, then pointing out the relation between the subdifferentiability of a function associated to a set of epigraph type at a certain point and the fact that a corresponding point is not in the quasi-relative interior of the closed convex hull of the set.
TL;DR: In this article, abstract codifferentiability, abstract quasidifferentiability and abstract convex (concave) approximations of a nonsmooth function mapping a topological vector space to an order complete topology vector lattice are introduced.
Abstract: In the article we use abstract convexity theory in order to unify and generalize many different concepts of nonsmooth analysis. We introduce the concepts of abstract codifferentiability, abstract quasidifferentiability and abstract convex (concave) approximations of a nonsmooth function mapping a topological vector space to an order complete topological vector lattice. We study basic properties of these notions, construct elaborate calculus of abstract codifferentiable functions and discuss continuity of abstract codifferential. We demonstrate that many classical concepts of nonsmooth analysis, such as subdifferentiability and quasidifferentiability, are particular cases of the concepts of abstract codifferentiability and abstract quasidifferentiability. We also show that abstract convex and abstract concave approximations are a very convenient tool for the study of nonsmooth extremum problems. We use these approximations in order to obtain various necessary optimality conditions for nonsmooth nonconvex o...
TL;DR: A descriptive experiment is designed to gain insight through impact of the novel heuristic, which reveals the heuristic fulfills both goals of an effective adaptation scheme: It reduces performance sensitivity to the control parameter and increases convergence efficiency at the same time.
Abstract: A simple, yet efficient scheme for adaptation of population size in evolution strategies (ESs) that utilize global intermediate/weighted recombination is presented. At the first step, a measure to quantify multimodality of the region under exploration is introduced. This quantity is iteratively updated based on the optimization history and subsequently utilized to enlarge the population size when facing highly multimodal regions and vice versa. A descriptive experiment is designed to gain insight through impact of the novel heuristic. The heuristic is incorporated into the Covariance Matrix Adaptation Evolution Strategy (CMA-ES), the state of the art evolution strategy, and the reinforced algorithm is compared to the basic CMA-ES with fixed population size on a large number of test problems. Result comparison reveals the heuristic fulfills both goals of an effective adaptation scheme: It reduces performance sensitivity to the control parameter and increases convergence efficiency at the same time.
TL;DR: In this article, the existence results for linear and pseudo-monotone variational inequalities in reflexive Banach spaces were gathered and a convergence algorithm for non-convex non-differentiable optimization problems was proposed.
Abstract: In this paper, we first gather existence results for linear and for pseudo-monotone variational inequalities in reflexive Banach spaces. We discuss the necessity of the involved coerciveness conditions and their relationship. Then, we combine Mosco convergence of convex closed sets with an approximation of pseudo-monotone bifunctions and provide a convergent approximation procedure for pseudo-monotone variational inequalities in reflexive Banach spaces. Since hemivariational inequalities in linear elasticity are pseudo-monotone, our approximation method applies to nonmonotone contact problems. We sketch how regularization of the involved nonsmooth functionals together with finite element approximation lead to an efficient numerical solution method for these nonconvex nondifferentiable optimization problems. To illustrate our theory, we give a numerical example of a 2D linear elastic block under a given nonmonotone contact law.
TL;DR: In this paper, necessary optimality conditions for variational problems with a Lagrangian depending on a combined Caputo derivative of variable fractional order are established, and the endpoint of the integral is free.
Abstract: We establish necessary optimality conditions for variational problems with a Lagrangian depending on a combined Caputo derivative of variable fractional order. The endpoint of the integral is free, and thus transversality conditions are proved. Several particular cases are considered illustrating the new results.
TL;DR: In this article, the authors proposed variants of the forward-backward splitting method for finding a zero of the sum of two operators, which is known to converge when the forward and the backward operators are monotone and with Lipschitz continuity of forward operator.
Abstract: In this paper, we propose variants of Forward-Backward splitting method for finding a zero of the sum of two operators. A classical modification of Forward-Backward method was proposed by Tseng, which is known to converge when the forward and the backward operators are monotone and with Lipschitz continuity of the forward operator. The conceptual algorithm proposed here improves Tseng’s method in some instances. The first and main part of our approach, contains an explicit Armijo-type search in the spirit of the extragradient-like methods for variational inequalities. During the iteration process, the search performs only one calculation of the forward-backward operator in each tentative of the step. This achieves a considerable computational saving when the forward-backward operator is computationally expensive. The second part of the scheme consists in special projection steps. The convergence analysis of the proposed scheme is given assuming monotonicity on both operators, without Lipschitz continuity ...
TL;DR: It is shown that this class of methods shares the known complexity properties of a simple steepest-descent scheme and that an approximate first-order critical point can be computed in at most function and gradient evaluations, where is the user-defined accuracy threshold on the gradient norm.
Abstract: The worst-case evaluation complexity of finding an approximate first-order critical point using gradient-related non-monotone methods for smooth non-convex and unconstrained problems is investigated. The analysis covers a practical linesearch implementation of these popular methods, allowing for an unknown number of evaluations of the objective function (and its gradient) per iteration. It is shown that this class of methods shares the known complexity properties of a simple steepest-descent scheme and that an approximate first-order critical point can be computed in at most ) function and gradient evaluations, where is the user-defined accuracy threshold on the gradient norm.
TL;DR: In this article, the weak convergence of a general gradient projection algorithm for minimizing a convex function of class over convex constraint set is studied in a Hilbertian framework, where the feasible set is approximated by half-spaces making projections explicit.
Abstract: In this paper, we first study in a Hilbertian framework the weak convergence of a general Gradient Projection Algorithm for minimizing a convex function of class over a convex constraint set. The way of selecting the stepsizes corresponds to the one used by Lopez et al. for the particular case of the Split Feasibility Problem. This choice allows us to avoid the computation of operator norms. Afterwards, a relaxed version of the Gradient Projection Algorithm is considered where the feasible set is approximated by half-spaces making the projections explicit. Finally, to get the strong convergence, each step of the general Gradient Projection Method is combined with a viscosity step. This is done by adapting Halpern’s algorithm to our problem. The general scheme is then applied to the Split Equality Problem, and also to the Split Feasibility Problem.