Numerical Optimization presents a comprehensive and up-to-date description of the most effective methods in continuous optimization. It responds to the growing interest in optimization in engineering, science, and business by focusing on the methods that are best suited to practical problems. For this new edition the book has been thoroughly updated throughout. There are new chapters on nonlinear interior methods and derivative-free methods for optimization, both of which are used widely in practice and the focus of much current research. Because of the emphasis on practical methods, as well as the extensive illustrations and exercises, the book is accessible to a wide audience. It can be used as a graduate text in engineering, operations research, mathematics, computer science, and business. It also serves as a handbook for researchers and practitioners in the field. The authors have strived to produce a text that is pleasant to read, informative, and rigorous - one that reveals both the beautiful nature of the discipline and its practical side.

Numerical Optimization

We consider the class of iterative shrinkage-thresholding algorithms (ISTA) for solving linear inverse problems arising in signal/image processing. This class of methods, which can be viewed as an extension of the classical gradient algorithm, is attractive due to its simplicity and thus is adequate for solving large-scale problems even with dense matrix data. However, such methods are also known to converge quite slowly. In this paper we present a new fast iterative shrinkage-thresholding algorithm (FISTA) which preserves the computational simplicity of ISTA but with a global rate of convergence which is proven to be significantly better, both theoretically and practically. Initial promising numerical results for wavelet-based image deblurring demonstrate the capabilities of FISTA which is shown to be faster than ISTA by several orders of magnitude.

http://people.rennes.inria.fr/Cedric.Herzet/Cedric.Herzet/Sparse_Seminar/Entrees/2012/11/12_A_Fast_Iterative_Shrinkage-Thresholding_Algorithmfor_Linear_Inverse_Problems_(A._Beck,_M._Teboulle)_files/Breck_2009.pdf

A Fast Iterative Shrinkage-Thresholding Algorithm for Linear Inverse Problems

List of Figures. List of Tables. Preface. Foreword. 1. Basic Concepts. 2. Evolutionary Algorithm MOP Approaches. 3. MOEA Test Suites. 4. MOEA Testing and Analysis. 5. MOEA Theory and Issues. 3. MOEA Theoretical Issues. 6. Applications. 7. MOEA Parallelization. 8. Multi-Criteria Decision Making. 9. Special Topics. 10. Epilog. Appendix A: MOEA Classification and Technique Analysis. Appendix B: MOPs in the Literature. Appendix C: Ptrue & PFtrue for Selected Numeric MOPs. Appendix D: Ptrue & PFtrue for Side-Constrained MOPs. Appendix E: MOEA Software Availability. Appendix F: MOEA-Related Information. Index. References.

/pdf/evolutionary-algorithms-for-solving-multi-objective-problems-2txxk4mqci.pdf

Evolutionary algorithms for solving multi-objective problems

/pdf/convex-analysisnoer-san-nojin-zhan-nituite-5dl2rbmoyr.pdf

Convex Analysisの二,三の進展について

In semidefinite programming, one minimizes a linear function subject to the constraint that an affine combination of symmetric matrices is positive semidefinite. Such a constraint is nonlinear and nonsmooth, but convex, so semidefinite programs are convex optimization problems. Semidefinite programming unifies several standard problems (e.g., linear and quadratic programming) and finds many applications in engineering and combinatorial optimization. Although semidefinite programs are much more general than linear programs, they are not much harder to solve. Most interior-point methods for linear programming have been generalized to semidefinite programs. As in linear programming, these methods have polynomial worst-case complexity and perform very well in practice. This paper gives a survey of the theory and applications of semidefinite programs and an introduction to primaldual interior-point methods for their solution.

Semidefinite programming

The paper considers stochastically constrained nonlinear programming problems. A penalty function is constructed in terms of a “distance” between random variables, defined in terms of the φ-divergence functional a generalization of the relative entropy. A duality theory is developed in which a general relation between φ-divergence and utility functions is revealed, via the conjugate transform, and a new type of certainty equivalent concept emerges.

Penalty Functions and Duality in Stochastic Programming Via φ-Divergence Functionals

Chapter Fourteen. Robust Adjustable Multistage Optimization

Free material design deals with the question of finding the stiffest structure with respect to one or more given loads which can be made when both the distribution of material and the material itself can freely vary. The case of one single load has been discussed in several recent papers, and an efficient numerical approach was presented in [M. Kocvara, M. Zibulevsky, and J. Zow, RAIRO Model. Math. Anal. Numer. 32 (1998), pp. 255--281]. We attack here the multiload situation (understood in the worst-case sense), which is of much more interest for applications but also significantly more challenging from both the theoretical and the numerical points of view. After a series of transformation steps we reach a problem formulation for which we can prove existence of a solution; a suitable discretization leads to a semidefinite programming problem for which modern polynomial time algorithms of interior point type are available. A number of numerical examples demonstrates the efficiency of our approach.

Free Material Design via Semidefinite Programming: The Multiload Case with Contact Conditions

We introduce a smoothing technique for nondifferentiable optimization problems. The approach is to replace the original problem by an approximate one which is controlled by a smoothing parameter. The recession function is instrumental in the construction of the approximate problem. An a priori bound on the difference between the optimal values of the original problem and the approximate one is explicitly derived in term of the smoothing parameter. The relationships between the primal approximated problem and its corresponding dual are investigated.

A smoothing technique for nondifferentiable optimization problems

Presents coincidence-list-ordered set expectation-maximisation (COSEM), an algorithm for iterative image reconstruction directly from list-mode coincidence acquisition data. The COSEM algorithm is based on the ordered sets EM algorithm for binned data but has several extensions that makes it suitable for rotating two planar detector tomographs. The authors develop the COSEM algorithm and extend it to include analytic calculation of detection probability, noise reducing iterative filtering schemes, and on-the-fly attenuation correction methods. The authors present an adaptation of COSEM to the VaricamVG camera and show results from clinical and phantom studies.

http://iew3.technion.ac.il/~mcib/cosemIeeeFinal.pdf

Aharon Ben-Tal

Papers

Penalty Functions and Duality in Stochastic Programming Via φ-Divergence Functionals

Chapter Fourteen. Robust Adjustable Multistage Optimization

Free Material Design via Semidefinite Programming: The Multiload Case with Contact Conditions

A smoothing technique for nondifferentiable optimization problems

The design and implementation of COSEN, an iterative algorithm for fully 3-D listmode data