Showing papers in "Journal of Optimization Theory and Applications in 1993"
TL;DR: In this article, the Lipschitz constant is viewed as a weighting parameter that indicates how much emphasis to place on global versus local search, which accounts for the fast convergence of the new algorithm on the test functions.
Abstract: We present a new algorithm for finding the global minimum of a multivariate function subject to simple bounds. The algorithm is a modification of the standard Lipschitzian approach that eliminates the need to specify a Lipschitz constant. This is done by carrying out simultaneous searches using all possible constants from zero to infinity. On nine standard test functions, the new algorithm converges in fewer function evaluations than most competing methods.
The motivation for the new algorithm stems from a different way of looking at the Lipschitz constant. In particular, the Lipschitz constant is viewed as a weighting parameter that indicates how much emphasis to place on global versus local search. In standard Lipschitzian methods, this constant is usually large because it must equal or exceed the maximum rate of change of the objective function. As a result, these methods place a high emphasis on global search and exhibit slow convergence. In contrast, the new algorithm carries out simultaneous searches using all possible constants, and therefore operates at both the global and local level. Once the global part of the algorithm finds the basin of convergence of the optimum, the local part of the algorithm quickly and automatically exploits it. This accounts for the fast convergence of the new algorithm on the test functions.
1,994 citations
TL;DR: This paper discusses five of these characteristics and presents a strategy for function optimization called the shuffled complex evolution (SCE) method, which promises to be robust, effective, and efficient for a broad class of problems.
Abstract: The degree of difficulty in solving a global optimization problem is in general dependent on the dimensionality of the problem and certain characteristics of the objective function. This paper discusses five of these characteristics and presents a strategy for function optimization called the shuffled complex evolution (SCE) method, which promises to be robust, effective, and efficient for a broad class of problems. The SCE method is based on a synthesis of four concepts that have proved successful for global optimization: (a) combination of probabilistic and deterministic approaches; (b) clustering; (c) systematic evolution of a complex of points spanning the space, in the direction of global improvement; and (d) competitive evolution. Two algorithms based on the SCE method are presented. These algorithms are tested by running 100 randomly initiated trials on eight test problems of differing difficulty. The performance of the two algorithms is compared to that of the controlled random search CRS2 method presented by Price (1983, 1987) and to a multistart algorithm based on the simplex method presented by Nelder and Mead (1965).
1,481 citations
TL;DR: In this paper, the problem of determining the circle of best fit to a set of points in the plane (or the obvious generalization ton-dimensions) is easily formulated as a nonlinear total least-squares problem which may be solved using a Gauss-Newton minimization algorithm.
Abstract: The problem of determining the circle of best fit to a set of points in the plane (or the obvious generalization ton-dimensions) is easily formulated as a nonlinear total least-squares problem which may be solved using a Gauss-Newton minimization algorithm. This straight-forward approach is shown to be inefficient and extremely sensitive to the presence of outliers. An alternative formulation allows the problem to be reduced to a linear least squares problem which is trivially solved. The recommended approach is shown to have the added advantage of being much less sensitive to outliers than the nonlinear least squares approach.
209 citations
TL;DR: A deterministic global optimization approach is proposed for nonconvex constrained nonlinear programming problems that converts the original problem into primal and relaxed dual subproblems that provide valid upper and lower bounds respectively on the global optimum.
Abstract: A deterministic global optimization approach is proposed for nonconvex constrained nonlinear programming problems. Partitioning of the variables, along with the introduction of transformation variables, if necessary, converts the original problem into primal and relaxed dual subproblems that provide valid upper and lower bounds respectively on the global optimum. Theoretical properties are presented which allow for a rigorous solution of the relaxed dual problem. Proofs of ∈-finite convergence and ∈-global optimality are provided. The approach is shown to be particularly suited to (a) quadratic programming problems, (b) quadratically constrained problems, and (c) unconstrained and constrained optimization of polynomial and rational polynomial functions. The theoretical approach is illustrated through a few example problems. Finally, some further developments in the approach are briefly discussed.
176 citations
TL;DR: In this paper, a non-monotonic trust region method for unconstrained optimization problems is presented, which allows the sequence of values of the objective function to be nonmonotone.
Abstract: A nonmonotonic trust region method for unconstrained optimization problems is presented. Although the method allows the sequence of values of the objective function to be nonmonotonic, convergence properties similar to those for the usual trust region method are proved under certain conditions, including conditions on the approximate solutions to the subproblem. To make the solution satisfy these conditions, an algorithm to solve the subproblem is also established. Finally, some numerical results are reported which show that the nonmonotonic trust region method is superior to the usual trust region method according to both the number of gradient evaluations and the number of function evaluations.
165 citations
TL;DR: This work describes interior point algorithms for a discrete-time linear-quadratic regulator problem with mixed state/control constraints and shows how they can be efficiently-incorporated into an inexact sequential quadratic programming algorithm for nonlinear problems.
Abstract: We show that recently developed interior point methods for quadratic programming and linear complementarity problems can be put to use in solving discrete-time optimal control problems, with general pointwise constraints on states and controls. We describe interior point algorithms for a discrete-time linear-quadratic regulator problem with mixed state/control constraints and show how they can be efficiently-incorporated into an inexact sequential quadratic programming algorithm for nonlinear problems. The key to the efficiency of the interior-point method is the narrow-banded structure of the coefficient matrix which is factorized at each iteration.
151 citations
TL;DR: In this paper, a formulation for identification of linear multivariable systems from single or multiple sets of input-output data is presented, where the observer is expressed in terms of an observer, which is made asymptotically stable by an embedded eigenvalue assignment procedure.
Abstract: This paper presents a formulation for identification of linear multivariable systems from single or multiple sets of input-output data. The system input-output relationship is expressed in terms of an observer, which is made asymptotically stable by an embedded eigenvalue assignment procedure. The prescribed eigenvalues for the observer may be real, complex, mixed real and complex, or zero corresponding to a deadbeat observer. In this formulation, the Markov parameters of the observer are first identified from input-output data. The Markov parameters of the actual system are then recovered from those of the observer and used to realize a state space model of the system. The basic mathematical formulation is derived, and numerical examples are presented to illustrate the proposed method.
139 citations
TL;DR: A new method for unconstrained global function optimization, acronymed TRUST, was introduced in this paper, which is based on the concept of global function optimizer and trust.
Abstract: A new method for unconstrained global function optimization, acronymed TRUST, is introduced.
138 citations
TL;DR: Numerical experiments indicate a significant advantage of the proposed line search over the Armijo search.
Abstract: It was recently shown that, in the solution of smooth constrained optimization problems by sequential quadratic programming (SQP), the Maratos effect can be prevented by means of a certain nonmonotone (more precisely, three-step or four-step monotone) line search Using a well-known transformation, this scheme can be readily extended to the case of minimax problems It turns out however that, due to the structure of these problems, one can use a simpler scheme Such a scheme is proposed and analyzed in this paper Numerical experiments indicate a significant advantage of the proposed line search over the Armijo search
133 citations
TL;DR: In this paper, it was shown that the general variational inequality problem is equivalent to solving the Wiener-Hopf equations, and a number of iterative algorithms for variational inequalities were proposed.
Abstract: In this paper, we show that the general variational inequality problem is equivalent to solving the Wiener-Hopf equations. We use this equivalence to suggest and analyze a number of iterative algorithms for solving general variational inequalities. We also discuss the convergence criteria for these algorithms.
128 citations
TL;DR: In this article, the authors derive first-order characterizations of generalized monotone maps based on a geometrical analysis of the generalized monotonicity of generalized convexity.
Abstract: This paper is a sequel to Ref. 1 in which several kinds of generalized monotonicity were introduced for maps. They were related to generalized convexity properties of functions in the case of gradient maps. In the present paper, we derive first-order characterizations of generalized monotone maps based on a geometrical analysis of generalized monotonicity. These conditions are both necessary and sufficient for generalized monotonicity. Specialized results are obtained for the affine case.
TL;DR: In this article, the authors consider the problem of stabilizing an uncertain system when the norm of the control input is bounded by a prespecified constant and present controllers which guarantee that, for all allowable uncertainties and nonlinearities, there is a region of attraction from which all solutions converge to the given ball with the pre-specified convergence rate.
Abstract: We consider the problem of stabilizing an uncertain system when the norm of the control input is bounded by a prespecified constant. We treat continuous-time dynamical systems whose nominal part is linear and whose uncertain part is norm-bounded by a known affine function of the norm of the system state and the norm of the control input. Given a prespecified rate of convergence and a ball containing the origin of the state space, we present controllers which guarantee that, for all allowable uncertainties and nonlinearities, there is a region of attraction from which all solutions converge to the given ball with the prespecified convergence rate.
TL;DR: In this paper, the existence of Pareto equilibria in multicriteria games is investigated in two ways: one follows the fixed-point technique, and the other utilizes other tools.
Abstract: In this paper, we investigate the existence of Pareto equilibria in multicriteria games. The investigation is carried out in two ways: one follows the fixed-point technique, and the other utilizes other tools. Several sufficient conditions are presented to guarantee the existence of a Pareto equilibrium.
TL;DR: In this article, a weak version of vector variational inequalities in ordered Banach spaces is introduced and dual cones are introduced and proved to be closed in the sense of efficient point.
Abstract: In this paper, vector complementarity problems are introduced as weak versions of vector variational inequalities in ordered Banach spaces. New dual cones are introduced and proved to be closed. In the sense of efficient point, we prove that the minimal element problem is solvable if a vector variational inequality is solvable; we also prove that any solution of a strong vector variational inequality or positive vector complementarity problem is a solution of the minimal element problem.
TL;DR: In this paper, a generalized vector variational inequality is considered, and its equivalence with a multi-objective programming problem is discussed using (α, φ,Q)-invexity.
Abstract: In this paper, (α, φ,Q)-invexity is introduced, where α:X ×X → intRm+, φ:X ×X →X,X is a Banach space,Q is a convex cone ofRm. This unifies the properties of many classes of functions, such asQ-convexity, pseudo-linearity, representation condition, null space condition, andV-invexity. A generalized vector variational inequality is considered, and its equivalence with a multi-objective programming problem is discussed using (α, φ,Q)-invexity. An existence theorem for the solution of a generalized vector variational inequality is proved. Some applications of (α, φ,Q)-invexity to multi-objective programming problems and to a special kind of generalized vector variational inequality are given.
TL;DR: In this paper, a class of functions called pseudo B-vex and quasi B -vex functions are introduced by relaxing the definitions of B-vesx, pseudoconvex, and quasiconvex.
Abstract: A class of functions called pseudo B-vex and quasi B-vex functions is introduced by relaxing the definitions of B-vex, pseudoconvex, and quasiconvex functions. Similarly, the class of B-invex, pseudo B-invex, and quasi B-invex functions is defined as a generalization of B-vex, pseudo B-vex, and quasi B-vex functions. The sufficient optimality conditions and duality results are obtained for a nonlinear programming problem involving B-vex and B-invex functions.
TL;DR: In this paper, first-order necessary optimality conditions are derived for a class of two-level Stackelberg problems in which the followers' lower-level problems are convex programs with unique solutions.
Abstract: First-order necessary optimality conditions are derived for a class of two-level Stackelberg problems in which the followers' lower-level problems are convex programs with unique solutions To this purpose, generalized Jacobians of the marginal maps corresponding to followers' problems are estimated As illustrative examples, two discretized optimum design problems with elliptic variational inequalities are investigated The theoretical results may be used also for the numerical solution of the Stackelberg problems considered by nondifferentiable optimization methods
TL;DR: In this article, complex pursuit-evasion games with state variable inequality constraints are investigated and necessary conditions of the first and second order for optimal trajectories are developed, which enable the calculation of optimal open-loop strategies.
Abstract: Complex pursuit-evasion games with state variable inequality constraints are investigated. Necessary conditions of the first and the second order for optimal trajectories are developed, which enable the calculation of optimal open-loop strategies. The necessary conditions on singular surfaces induced by state constraints and non-smooth data are discussed in detail. These conditions lead to multi-point boundary-value problems which can be solved very efficiently and very accurately by the multiple shooting method. A realistically modelled pursuit-evasion problem for one air-to-air missile versus one high performance aircraft in a vertical plane serves as an example. For this pursuit-evasion game, the barrier surface is investigated, which determines the firing range of the missile. The numerical method for solving this problem and extensive numerical results will be presented and discussed in Part 2 of this paper; see Ref. 1.
TL;DR: In this article, the authors characterize a class of feasible direction methods in nonlinear programming through the concept of partial linearization of the objective function, where the objective is replaced by an arbitrary convex and continuously differentiable function, and the error is taken into account by a first-order approximation of it.
Abstract: In this paper, we characterize a class of feasible direction methods in nonlinear programming through the concept of partial linearization of the objective function. Based on a feasible point, the objective is replaced by an arbitrary convex and continuously differentiable function, and the error is taken into account by a first-order approximation of it. A new feasible point is defined through a line search with respect to the original objective, toward the solution of the approximate problem. Global convergence results are given for exact and approximate line searches, and possible interpretations are made. We present some instances of the general algorithm and discuss extensions to nondifferentiable programming.
TL;DR: In this paper, a class of functions called B-preinvex functions is introduced by relaxing the definitions of pre-inverse and B-vex, and some properties of B-Preinverse functions are obtained.
Abstract: A class of functions called B-preinvex functions is introduced by relaxing the definitions of preinvex and B-vex functions. Examples are given to show that there exist functions which are B-preinvex but not preinvex or B-vex or quasipreinvex. Some of the properties of B-preinvex functions are obtained.
TL;DR: In this article, the problem of finding a control strategy for a linear discrete-time periodic system with state and control bounds in the presence of unknown disturbances that are only known to belong to a given compact set is considered.
Abstract: The problem is considered of finding a control strategy for a linear discrete-time periodic system with state and control bounds in the presence of unknown disturbances that are only known to belong to a given compact set. This kind of problem arises in practice in resource distribution systems where the demand has typically a periodic behavior, but cannot be estimated a priori without an uncertainty margin. An infinite-horizon keeping problem is formulated, which consists in confining the state within its constraint set using the allowable control, whatever the allowed disturbances may be. To face this problem, the concepts of periodically invariant set and sequence are introduced. They are used to formulate a solution strategy that solves the keeping problem. For the case of polyhedral state, control, and disturbance constraints, a computationally feasible procedure is proposed. In particular, it is shown that periodically invariant sequences may be computed off-line, and then they may be used to synthesize on-line a control strategy. Finally, an optimization criterion for the control law is discussed.
TL;DR: In this paper, the necessary and sufficient conditions for a Pareto optimum of a subdifferentiable multiobjective fractional programming problem are derived without recourse to an equivalent convex program or parametric transformation.
Abstract: Fritz John and Kuhn-Tucker necessary and sufficient conditions for a Pareto optimum of a subdifferentiable multiobjective fractional programming problem are derived without recourse to an equivalent convex program or parametric transformation. A dual problem is introduced and, under convexity assumptions, duality theorems are proved. Furthermore, a Lagrange multiplier theorem is established, a vector-valued ratio-type Lagrangian is introduced, and vector-valued saddle-point results are presented.
TL;DR: In this paper, necessary conditions for optimal open-loop strategies in differential games of pursuit-evasion type have been developed for problems which involve state variable inequality constraints and nonsmooth data.
Abstract: In Part 1 of this paper (Ref. 1), necessary conditions for optimal open-loop strategies in differential games of pursuit-evasion type have been developed for problems which involve state variable inequality constraints and nonsmooth data. These necessary conditions lead to multipoint boundary-value problems with jump conditions. These problems can be solved very efficiently and accurately by the well-known multiple-shooting method. By this approach, optimal open-loop strategies and their associated saddle-point trajectories can be computed for the entire capture zone of the game. This also includes the computation of optimal open-loop strategies and saddle-point trajectories on the barrier of the pursuit-evasion game. The open-loop strategies provide an open-loop representation of the optimal feedback strategies. Numerical results are obtained for a special air combat scenario between one medium-range air-to-air missile and one high-performance aircraft in a vertical plane. A dynamic pressure limit for the aircraft imposes a state variable inequality constraint of the first order. Special emphasis is laid on realistic approximations of the lift, drag, and thrust of both vehicles and the atmospheric data. In particular, saddle-point trajectories on the barrier are computed and discussed. Submanifolds of the barrier which separate the initial values of the capture zone from those of the escape zone are computed for two representative launch positions of the missible. By this way, the firing range of the pursuing missile is determined and visualized.
TL;DR: In this paper, a parametrized convex vector optimization problem with a parameter vectoru is considered and sufficient conditions for MinDW=MinDY andDW=W minDY are obtained, respectively.
Abstract: We consider a parametrized convex vector optimization problem with a parameter vectoru. LetY(u) be the objective space image of the parametrized feasible region. The perturbation mapW(u) is defined as the set of all minimal points of the setY(u) with respect to an ordering cone in the objective space. The purpose of this paper is to investigate the relationship between the contingent derivativeDW ofW and the contingent derivativeDY ofY. Sufficient conditions for MinDW=MinDY andDW=W minDY are obtained, respectively. Therefore, quantitative information on the behavior of the perturbation map is provided.
TL;DR: This paper analyzes a constrained optimization algorithm that combines an unconstrained minimization scheme like the conjugate gradient method, an augmented Lagrangian, and multiplier updates to obtain global quadratic convergence.
Abstract: This paper analyzes a constrained optimization algorithm that combines an unconstrained minimization scheme like the conjugate gradient method, an augmented Lagrangian, and multiplier updates to obtain global quadratic convergence. Some of the issues that we focus on are the treatment of rigid constraints that must be satisfied during the iterations and techniques for balancing the error associated with constraint violation with the error associated with optimality. A preconditioner is constructed with the property that the rigid constraints are satisfied while ill-conditioning due to penalty terms is alleviated. Various numerical linear algebra techniques required for the efficient implementation of the algorithm are presented, and convergence behavior is illustrated in a series of numerical experiments.
TL;DR: In this paper, it was shown that strongly pseudoconvex functions do not necessarily have a strongly pseudo-convex gradient, and that a strongly pseudocvex function which is only once but not twice differentiable does not have a strong pseudocovex gradient.
Abstract: An example in Ref. 1 is corrected to show that indeed a strongly pseudoconvex function, which is only once but not twice differentiable, does not necessarily have a strongly pseudoconvex gradient.
TL;DR: In this article, a class of infinite-horizon discounted optimal control problems with nonsmooth problem data is considered and a maximum principle in terms of differential inclusions with a Michel type transversality condition is given.
Abstract: In this paper, we consider a class of infinite-horizon discounted optimal control problems with nonsmooth problem data. A maximum principle in terms of differential inclusions with a Michel type transversality condition is given. It is shown that, when the discount rate is sufficiently large, the problem admits normal multipliers and a strong transversality condition holds. A relationship between dynamic programming and the maximum principle is also given.
TL;DR: In this article, the convergence of sets and functions in possibly infinite-dimensional spaces is derived from two results dealing with the continuity with respect to a parameter of the intersection of two convex sets depending on this parameter.
Abstract: New results about convergence of sets and functions in possibly infinite-dimensional spaces are derived in a simple and unified way from two results dealing with the continuity with respect to a parameter of the intersection of two convex sets depending on this parameter.
TL;DR: In this paper, Lagrange multipliers useful in characterizations of solutions to spectral estimation problems are proved to exist in the absence of Slater's condition provided a new constraint involving the quasi-relative interior holds.
Abstract: Lagrange multipliers useful in characterizations of solutions to spectral estimation problems are proved to exist in the absence of Slater's condition provided a new constraint involving the quasi-relative interior holds We also discuss the quasi interior and its relation to other generalizations of the interior of a convex set and relationships between various constraint qualifications Finally, we characterize solutions to theLp spectral estimation problem with the added constraint that the feasible vectors lie in a measurable strip [α, β]
TL;DR: In this article, a damped Gauss-Newton algorithm was proposed to solve the complementarity problem, which is equivalent to the solution of a system of nonlinear equations.
Abstract: Mangasarian has shown that the solution of the complementarity problem is equivalent to the solution of a system of nonlinear equations. In this paper, we propose a damped Gauss-Newton algorithm to solve this system, prove that under appropriate hypotheses one gets rapid local convergence, and present computational experience.