This paper shows, by means of an operator called asplitting operator, that the Douglas--Rachford splitting method for finding a zero of the sum of two monotone operators is a special case of the proximal point algorithm. Therefore, applications of Douglas--Rachford splitting, such as the alternating direction method of multipliers for convex programming decomposition, are also special cases of the proximal point algorithm. This observation allows the unification and generalization of a variety of convex programming algorithms. By introducing a modified version of the proximal point algorithm, we derive a new,generalized alternating direction method of multipliers for convex programming. Advances of this sort illustrate the power and generality gained by adopting monotone operator theory as a conceptual framework.

/pdf/on-the-douglas-rachford-splitting-method-and-the-proximal-1bdzwtx9b7.pdf

On the Douglas-Rachford splitting method and the proximal point algorithm for maximal monotone operators

For variational problems of the form Inf v∈ V {f(Av)+g(v)} , we propose a dual method which decouples the difficulties relative to the functionals f and g from the possible ill-conditioning effects of the linear operator A. The approach is based on the use of an Augmented Lagrangian functional and leads to an efficient and simply implementable algorithm. We study also the finite element approximation of such problems, compatible with the use of our algorithm. The method is finally applied to solve several problems of continuum mechanics.

A dual algorithm for the solution of nonlinear variational problems via finite element approximation

This paper presents an overview of our most recent results concerning the Particle Swarm Optimization (PSO) method. Techniques for the alleviation of local minima, and for detecting multiple minimizers are described. Moreover, results on the ability of the PSO in tackling Multiobjective, Minimax, Integer Programming and e1 errors-in-variables problems, as well as problems in noisy and continuously changing environments, are reported. Finally, a Composite PSO, in which the heuristic parameters of PSO are controlled by a Differential Evolution algorithm during the optimization, is described, and results for many well-known and widely used test functions are given.

Recent approaches to global optimization problems through Particle Swarm Optimization

The theory of the proximal point algorithm for maximal monotone operators is applied to three algorithms for solving convex programs, one of which has not previously been formulated Rate-of-convergence results for the “method of multipliers,” of the strong sort already known, are derived in a generalized form relevant also to problems beyond the compass of the standard second-order conditions for oplimality The new algorithm, the “proximal method of multipliers,” is shown to have much the same convergence properties, but with some potential advantages

https://www.jstor.org/stable/pdfplus/3689277.pdf

Augmented Lagrangians and Applications of the Proximal Point Algorithm in Convex Programming

An insightful, concise, and rigorous treatment of the basic theory of convex sets and functions in finite dimensions, and the Dual problem the feasible if it is they. Subgradient methods applied mathematics and sofware full. Ellipsoid method frankwolfe for publication. Arg max are the special case when choosing such. Unlike some convex programming lp a candidate solutions is they possess multiple to start! Operations research because this method which one would want. However for a project that lie. Classical optimization problem of agents that converge. For publication another criterion for this may not dominated by far. Gradient methods are some of applied to optimization problems may. The conditions using the objective function is a final. Arg max are allowed set of non convex course. This finite time average of convex sets can. Convexity theory convex if it can be efficiently and algorithms proposed for classes. The book is not distinguish maxima, are even harder to a large. However it is not refer to relax the hessian matrix in terms of linear programming. Present the problem of making usually, much slower than modern. Some combinatorial optimization and increasingly popular method but not done by the use divergent series. For the supremum operator for every equality constraint manifold dimension. The drift plus penalty method for many optimization. The problem itself which the class of hessians.

/pdf/convex-optimization-theory-iix1w84r2j.pdf

Convex Optimization Theory

During recent years it has been shown that the performance of penalty function methods for constrained minimization can be improved significantly by introducing gradient type iterations for solving the dual problem. In this paper we present a new penalty function algorithm of this type which offers significant advantages over existing schemes for the case of the convex programming problem. The algorithm treats inequality constraints explicitly and can also be used for the solution of general mathematical programming problems.

A new penalty function method for constrained minimization

In this paper we propose and analyze a class of combined primal–dual and penalty methods for constrained minimization which generalizes the method of multipliers. We provide a convergence and rate of convergence analysis for these methods for the case of a convex programming problem. We prove global convergence in the presence of both exact and inexact unconstrained minimization, and we show that the rate of convergence may be linear or superlinear with arbitrary Q-order of convergence depending on the problem at hand and the form of the penalty function employed.

Barry W. Kort

Papers

A new penalty function method for constrained minimization

Combined Primal–Dual and Penalty Methods for Convex Programming