/pdf/convex-analysisnoer-san-nojin-zhan-nituite-5dl2rbmoyr.pdf

Convex Analysisの二,三の進展について

https://www.maths.ed.ac.uk/~gondzio/reports/ipmXXV.pdf

Interior point methods 25 years later

In this paper we present a redesign of a linear algebra kernel of an interior point method to avoid the explicit use of problem matrices. The only access to the original problem data needed are the matrix-vector multiplications with the Hessian and Jacobian matrices. Such a redesign requires the use of suitably preconditioned iterative methods and imposes restrictions on the way the preconditioner is computed. A two-step approach is used to design a preconditioner. First, the Newton equation system is regularized to guarantee better numerical properties and then it is preconditioned. The preconditioner is implicit, that is, its computation requires only matrix-vector multiplications with the original problem data. The method is therefore well-suited to problems in which matrices are not explicitly available and/or are too large to be stored in computer memory. Numerical properties of the approach are studied including the analysis of the conditioning of the regularized system and that of the preconditioned regularized system. The method has been implemented and preliminary computational results for small problems limited to 1 million of variables and 10 million of nonzero elements demonstrate the feasibility of the approach.

/pdf/matrix-free-interior-point-method-2c52fqvfx2.pdf

Matrix-free interior point method

On the steplength selection in gradient methods for unconstrained optimization

The solution of KKT systems is ubiquitous in optimization methods and often dominates the computation time, especially when large-scale problems are considered. Thus, the effective implementation of such methods is highly dependent on the availability of effective linear algebra algorithms and software, that are able, in turn, to take into account specific needs of optimization. In this paper we discuss the mutual impact of linear algebra and optimization, focusing on interior point methods and on the iterative solution of the KKT system. Three critical issues are addressed: preconditioning, termination control for the inner iterations, and inertia control.

/pdf/on-mutual-impact-of-numerical-linear-algebra-and-large-scale-4cbblsx216.pdf

On mutual impact of numerical linear algebra and large-scale optimization with focus on interior point methods

Iterative solvers appear to be very promising in the development of efficient software, based on Interior Point methods, for large-scale nonlinear optimization problems. In this paper we focus on the use of preconditioned iterative techniques to solve the KKT system arising at each iteration of a Potential Reduction method for convex Quadratic Programming. We consider the augmented system approach and analyze the behaviour of the Constraint Preconditioner with the Conjugate Gradient algorithm. Comparisons with a direct solution of the augmented system and with MOSEK show the effectiveness of the iterative approach on large-scale sparse problems.

On the iterative solution of KKT systems in potential reduction software for large-scale quadratic problems

We present a technique for building effective and low cost preconditioners for sequences of shifted linear systems $(A + \alpha I) x_\alpha = b$, where $A$ is symmetric positive definite and $\alpha > 0$. This technique updates a preconditioner for $A$, available in the form of an $LDL^T$ factorization, by modifying only the nonzero entries of the $L$ factor in such a way that the resulting preconditioner mimics the diagonal of the shifted matrix and reproduces its overall behavior. This approach is supported by a theoretical analysis as well as by numerical experiments, showing that it works efficiently for a broad range of values of $\alpha$.

Efficient Preconditioner Updates for Shifted Linear Systems

We propose a framework for building preconditioners for sequences of linear systems of the form $(A+\Delta_k) x_k=b_k$, where $A$ is symmetric positive semidefinite and $\Delta_k$ is diagonal positive semidefinite. Such sequences arise in several optimization methods, e.g., in affine-scaling methods for bound-constrained convex quadratic programming and bound-constrained linear least squares, as well as in trust-region and overestimation methods for convex unconstrained optimization problems and nonlinear least squares. For all the matrices of a sequence, the preconditioners are obtained by updating any preconditioner for $A$ available in the $LDL^T$ form. The preconditioners in the framework satisfy the natural requirement of being effective on slowly varying sequences; furthermore, under an additional property they are also able to cluster eigenvalues of the preconditioned matrix when some entries of $\Delta_k$ are sufficiently large. We present two low-cost preconditioners sharing the above-mentioned p...

A Preconditioning Framework for Sequences of Diagonally Modified Linear Systems Arising in Optimization

We investigate the application of the nonmonotone spectral projected gradient (SPG) method to a region-based variational model for image segmentation. We consider a "discretize-then-optimize" approach and solve the resulting nonlinear optimization problem by an alternating minimization procedure that exploits the SPG2 algorithm by Birgin et al. (SIAM J Optim 10(4):1196---1211, 2000). We provide a convergence analysis and perform numerical experiments on several images, showing the effectiveness of this procedure.

/pdf/on-the-application-of-the-spectral-projected-gradient-method-2dtky8thjw.pdf

Valentina De Simone

Papers

On mutual impact of numerical linear algebra and large-scale optimization with focus on interior point methods

On the iterative solution of KKT systems in potential reduction software for large-scale quadratic problems

Efficient Preconditioner Updates for Shifted Linear Systems

A Preconditioning Framework for Sequences of Diagonally Modified Linear Systems Arising in Optimization

On the Application of the Spectral Projected Gradient Method in Image Segmentation