Journal ArticleDOI
Recent advances in trust region algorithms
TLDR
Recent results on trust region methods for unconstrained optimization, constrained optimization, nonlinear equations and nonlinear least squares, nonsmooth optimization and optimization without derivatives are reviewed.Abstract:
Trust region methods are a class of numerical methods for optimization. Unlike line search type methods where a line search is carried out in each iteration, trust region methods compute a trial step by solving a trust region subproblem where a model function is minimized within a trust region. Due to the trust region constraint, nonconvex models can be used in trust region subproblems, and trust region algorithms can be applied to nonconvex and ill-conditioned problems. Normally it is easier to establish the global convergence of a trust region algorithm than that of its line search counterpart. In the paper, we review recent results on trust region methods for unconstrained optimization, constrained optimization, nonlinear equations and nonlinear least squares, nonsmooth optimization and optimization without derivatives. Results on trust region subproblems and regularization methods are also discussed.read more
Citations
More filters
Journal ArticleDOI
Non-convex Optimization for Machine Learning
Prateek Jain,Purushottam Kar +1 more
TL;DR: Non-convex optimization as discussed by the authors is a generalization of the convex optimization problem, and it has been widely used in machine learning applications, such as deep learning and reinforcement learning.
Book
Non-convex Optimization for Machine Learning
Prateek Jain,Purushottam Kar +1 more
TL;DR: Non-convex optimization as mentioned in this paper is a popular approach for large-scale optimization problems and has been shown to outperform relaxation-based heuristics, such as projected gradient descent and alternating minimization.
Journal ArticleDOI
Data-Driven Short-Term Solar Irradiance Forecasting Based on Information of Neighboring Sites
Chao Huang,Long Wang,Loi Lei Lai +2 more
TL;DR: A data-driven framework for forecasting solar irradiance based on fusing spatial and temporal information is proposed and it is demonstrated that the BRT model gives the best performance with the lowest normalized root mean squared error.
Journal ArticleDOI
Complexity and global rates of trust-region methods based on probabilistic models
TL;DR: The derivation of the global rates and worst case complexity bounds follows closely from a study of direct-search methods based on the companion notion of probabilistic descent, providing both first and second order measures of optimality.
Journal ArticleDOI
Trust-Region Methods Without Using Derivatives: Worst Case Complexity and the NonSmooth Case
TL;DR: This paper starts by analyzing the worst case complexity of general trust-region derivative-free methods for smooth functions, and proposes a smoothing approach, for the nonsmooth case, for which it is shown how to improve the existing resu...
References
More filters
Book
Numerical Optimization
Jorge Nocedal,Stephen J. Wright +1 more
TL;DR: Numerical Optimization presents a comprehensive and up-to-date description of the most effective methods in continuous optimization, responding to the growing interest in optimization in engineering, science, and business by focusing on the methods that are best suited to practical problems.
Journal ArticleDOI
A method for the solution of certain non – linear problems in least squares
TL;DR: In this article, the problem of least square problems with non-linear normal equations is solved by an extension of the standard method which insures improvement of the initial solution, which can also be considered an extension to Newton's method.
Book
Optimization and nonsmooth analysis
TL;DR: The Calculus of Variations as discussed by the authors is a generalization of the calculus of variations, which is used in many aspects of analysis, such as generalized gradient descent and optimal control.