Numerical Optimization presents a comprehensive and up-to-date description of the most effective methods in continuous optimization. It responds to the growing interest in optimization in engineering, science, and business by focusing on the methods that are best suited to practical problems. For this new edition the book has been thoroughly updated throughout. There are new chapters on nonlinear interior methods and derivative-free methods for optimization, both of which are used widely in practice and the focus of much current research. Because of the emphasis on practical methods, as well as the extensive illustrations and exercises, the book is accessible to a wide audience. It can be used as a graduate text in engineering, operations research, mathematics, computer science, and business. It also serves as a handbook for researchers and practitioners in the field. The authors have strived to produce a text that is pleasant to read, informative, and rigorous - one that reveals both the beautiful nature of the discipline and its practical side.

Numerical Optimization

We consider the class of iterative shrinkage-thresholding algorithms (ISTA) for solving linear inverse problems arising in signal/image processing. This class of methods, which can be viewed as an extension of the classical gradient algorithm, is attractive due to its simplicity and thus is adequate for solving large-scale problems even with dense matrix data. However, such methods are also known to converge quite slowly. In this paper we present a new fast iterative shrinkage-thresholding algorithm (FISTA) which preserves the computational simplicity of ISTA but with a global rate of convergence which is proven to be significantly better, both theoretically and practically. Initial promising numerical results for wavelet-based image deblurring demonstrate the capabilities of FISTA which is shown to be faster than ISTA by several orders of magnitude.

http://people.rennes.inria.fr/Cedric.Herzet/Cedric.Herzet/Sparse_Seminar/Entrees/2012/11/12_A_Fast_Iterative_Shrinkage-Thresholding_Algorithmfor_Linear_Inverse_Problems_(A._Beck,_M._Teboulle)_files/Breck_2009.pdf

A Fast Iterative Shrinkage-Thresholding Algorithm for Linear Inverse Problems

List of Figures. List of Tables. Preface. Foreword. 1. Basic Concepts. 2. Evolutionary Algorithm MOP Approaches. 3. MOEA Test Suites. 4. MOEA Testing and Analysis. 5. MOEA Theory and Issues. 3. MOEA Theoretical Issues. 6. Applications. 7. MOEA Parallelization. 8. Multi-Criteria Decision Making. 9. Special Topics. 10. Epilog. Appendix A: MOEA Classification and Technique Analysis. Appendix B: MOPs in the Literature. Appendix C: Ptrue & PFtrue for Selected Numeric MOPs. Appendix D: Ptrue & PFtrue for Side-Constrained MOPs. Appendix E: MOEA Software Availability. Appendix F: MOEA-Related Information. Index. References.

/pdf/evolutionary-algorithms-for-solving-multi-objective-problems-2txxk4mqci.pdf

Evolutionary algorithms for solving multi-objective problems

/pdf/convex-analysisnoer-san-nojin-zhan-nituite-5dl2rbmoyr.pdf

Convex Analysisの二,三の進展について

In semidefinite programming, one minimizes a linear function subject to the constraint that an affine combination of symmetric matrices is positive semidefinite. Such a constraint is nonlinear and nonsmooth, but convex, so semidefinite programs are convex optimization problems. Semidefinite programming unifies several standard problems (e.g., linear and quadratic programming) and finds many applications in engineering and combinatorial optimization. Although semidefinite programs are much more general than linear programs, they are not much harder to solve. Most interior-point methods for linear programming have been generalized to semidefinite programs. As in linear programming, these methods have polynomial worst-case complexity and perform very well in practice. This paper gives a survey of the theory and applications of semidefinite programs and an introduction to primaldual interior-point methods for their solution.

Semidefinite programming

We consider the optimization problem 

$$ \begin{array}{*{20}{c}} {\inf \,f(q,x)} \\ {subject\,to\,x \in \Omega (q)} \\ {q \in w;} \end{array} $$

(1.1)

here q ∈ℝ n , x ∈ℝ m , f [ℝ n × ℝ m → ℝ] is continuously differentiate, Ω [ℝ n ⤳ ℝ m ] is a proper closed-valued multifunction and ψ is a nonempty compact subset of ℝ n . Problem (1.1) differs from the classical optimal control model due the multivaluedness of Ω which can be now viewed as a multivalued system map.

A Nondifferentiable Approach to Decomposable Optimization Problems with an Application to the Design of Water Distribution Networks

With respect to a given family of functions 9, a function is said to be ^-convex, if it is supported, at each point, by some member of 9. For particular choices of 9 one obtains the convex functions and the generalized convex functions in the sense of Beckenbach. ^-convex functions are characterized and studied, retaining some essential results of classical convexity.

/pdf/a-generalization-of-convex-functions-via-support-properties-12amhfaru8.pdf

A generalization of convex functions via support properties

Maximizing a convex function over convex constraints is an NP-hard problem in general. We prove that such a problem can be reformulated as an adjustable robust optimization (ARO) problem in which each adjustable variable corresponds to a unique constraint of the original problem. We use ARO techniques to obtain approximate solutions to the convex maximization problem. In order to demonstrate the complete approximation scheme, we distinguish the cases in which we have just one nonlinear constraint and multiple linear constraints. Concerning the first case, we give three examples in which one can analytically eliminate the adjustable variable and approximately solve the resulting static robust optimization problem efficiently. More specifically, we show that the norm constrained log-sum-exp (geometric) maximization problem can be approximated by (convex) exponential cone optimization techniques. Concerning the second case of multiple linear constraints, the equivalent ARO problem can be represented as an adjustable robust linear optimization problem. Using linear decision rules then returns a safe approximation of the constraints. The resulting problem is a convex optimization problem, and solving this problem gives an upper bound on the global optimum value of the original problem. By using the optimal linear decision rule, we obtain a lower bound solution as well. We derive the approximation problems explicitly for quadratic maximization, geometric maximization, and sum-of-max-linear-terms maximization problems with multiple linear constraints. Numerical experiments show that, contrary to the state-of-the-art solvers, we can approximate large-scale problems swiftly with tight bounds. In several cases, we have equal upper and lower bounds, which concludes that we have global optimality guarantees in these cases. Summary of Contribution: Maximizing a convex function over a convex set is a hard optimization problem. We reformulate this problem as an optimization problem under uncertainty, which allows us to transfer the hardness of this problem from its nonconvexity to the uncertainty of the new problem. The equivalent uncertain optimization problem can be relaxed tightly by using adjustable robust optimization techniques. In addition to building a new bridge between convex maximization and robust optimization, this approach also gives us strong algorithms that improve the state-of-the-art optimization solvers both in solution time and quality for various convex maximization problems.

Convex Maximization via Adjustable Robust Optimization

We construct a mechanism to generate a large class of duality schemes for (not necessarily differentiable) convex optimization problems, for which the dual problem is continuously differentiable. We use the conjugate duality framework of Rockafellar; the original primal problem is embedded in a family of perturbed problems. The perturbation function is constructed in terms of two perturbation vectors and a single-variable function $q$. The differentiability is a consequence of the dual objective function's being a kind of ``proximal regularization," but one which is expressed in terms of a {\em nonquadratic} regularizing term associated with the function $q$.

A Conjugate Duality Scheme Generating a New Class of Differentiable Duals

For a given set of k constraints which depend on a real parameter x, $x \in [0,1]$ (or any other interval) it is desired to find the maximal subinterval $[0,\lambda ]$ where all these constraints are satisfied. In order to find $\lambda $ one can make n sequential observations where each of them can be made on any one of the constraints at a chosen point.If $k = 1$ then the minimax procedure (bisection) guarantees an accuracy of $(\frac{1}{2})^n $ in locating $\lambda $. The methods considered in this paper are applicable for $k > 1$. The surprising property of some of these methods is the fact that for any k, they guarantee an accuracy of $(\frac{1}{2} + \varepsilon _k )^n $ where $\varepsilon _k $ rapidly decreases to zero as n tends to infinity. Thus, for large n the asymptotic behavior is “almost” like the bisection procedure for one constraint. Numerical experiments show that these methods are efficient even for relatively small n and that they can handle effectively any number of constraints.

Aharon Ben-Tal

Papers

A Nondifferentiable Approach to Decomposable Optimization Problems with an Application to the Design of Water Distribution Networks

A generalization of convex functions via support properties

Convex Maximization via Adjustable Robust Optimization

A Conjugate Duality Scheme Generating a New Class of Differentiable Duals

On Finding the Maximal Range of Validity of a Constrained System