Journal ArticleDOI
The Barzilai and Borwein Gradient Method for the Large Scale Unconstrained Minimization Problem
TLDR
Results indicate that the global Barzilai and Borwein method may allow some significant reduction in the number of line searches and also in theNumber of gradient evaluations.Abstract:
The Barzilai and Borwein gradient method for the solution of large scale unconstrained minimization problems is considered. This method requires few storage locations and very inexpensive computations. Furthermore, it does not guarantee descent in the objective function and no line search is required. Recently, the global convergence for the convex quadratic case has been established. However, for the nonquadratic case, the method needs to be incorporated in a globalization scheme. In this work, a nonmonotone line search strategy that guarantees global convergence is combined with the Barzilai and Borwein method. This strategy is based on the nonmonotone line search technique proposed by Grippo, Lampariello, and Lucidi [SIAM J. Numer. Anal., 23 (1986), pp. 707--716]. Numerical results to compare the behavior of this method with recent implementations of the conjugate gradient method are presented. These results indicate that the global Barzilai and Borwein method may allow some significant reduction in the number of line searches and also in the number of gradient evaluations.read more
Citations
More filters
Journal ArticleDOI
Approximated Function Based Spectral Gradient Algorithm for Sparse Signal Recovery
Weifeng Wang,Qiuyu Wang +1 more
TL;DR: This paper constructs a new surrogate function to approximate l0-norm regularization, and subsequently makes the discrete optimization problem continuous and smooth, and uses the well-known spectral gradient algorithm to solve the resulting smooth optimization problem.
Proceedings ArticleDOI
A new regularization method for bi-level image restoration
Jianjun Zhang,Qin Wang +1 more
TL;DR: Zhang et al. as mentioned in this paper proposed a new regularization method for bi-level image restoration, which leads to a nonlinear optimization problem, which is solved by using the global Barzilai and Borwein gradient method.
Journal ArticleDOI
Nonsmooth spectral gradient methods for unconstrained optimization
TL;DR: This work combines the spectral choice of step length with two well-established subdifferential-type schemes: the gradient sampling method and the simplex gradient method to solve nonsmooth unconstrained minimization problems.
Proceedings ArticleDOI
On Descent Spectral CG Algorithms for Training Recurrent Neural Networks
TL;DR: The presented algorithm preserves the advantages of classical conjugate gradient methods while simultaneously avoids the usually inefficient restarts and proposes a new algorithm for training recurrent neural networks.
Accelerated gradient methods for total-variation-based CT image reconstruction
Jakob Heide Jørgensen,Tobias Lindstrøm Jensen,Per Christian Hansen,Søren Holdt Jensen,Emil Y. Sidky,Xiaochuan Pan +5 more
TL;DR: In this article, two accelerated gradient-based methods, GPBB and UPN, are proposed to solve the 3D-TV minimization problem in CT image reconstruction, which is based on Barzilai-Borwein (BB) step size selection and non-monotone line search.
References
More filters
Book
Practical Methods of Optimization
TL;DR: The aim of this book is to provide a Discussion of Constrained Optimization and its Applications to Linear Programming and Other Optimization Problems.
Book
Numerical Methods for Unconstrained Optimization and Nonlinear Equations (Classics in Applied Mathematics, 16)
TL;DR: In this paper, Schnabel proposed a modular system of algorithms for unconstrained minimization and nonlinear equations, based on Newton's method for solving one equation in one unknown convergence of sequences of real numbers.
Book
Numerical methods for unconstrained optimization and nonlinear equations
TL;DR: Newton's Method for Nonlinear Equations and Unconstrained Minimization and methods for solving nonlinear least-squares problems with Special Structure.
Journal ArticleDOI
Two-Point Step Size Gradient Methods
TL;DR: Etude de nouvelles methodes de descente suivant le gradient for the solution approchee du probleme de minimisation sans contrainte. as mentioned in this paper.