scispace - formally typeset
Search or ask a question

Showing papers on "Maxima and minima published in 1999"


Journal ArticleDOI
TL;DR: The stochastic approximation EM (SAEM), which replaces the expectation step of the EM algorithm by one iteration of a stochastics approximation procedure, is introduced and it is proved that, under mild additional conditions, the attractive stationary points of the SAEM algorithm correspond to the local maxima of the function.
Abstract: The expectation-maximization (EM) algorithm is a powerful computational technique for locating maxima of functions. It is widely used in statistics for maximum likelihood or maximum a posteriori estimation in incomplete data models. In certain situations, however, this method is not applicable because the expectation step cannot be performed in closed form. To deal with these problems, a novel method is introduced, the stochastic approximation EM (SAEM), which replaces the expectation step of the EM algorithm by one iteration of a stochastic approximation procedure. The convergence of the SAEM algorithm is established under conditions that are applicable to many practical situations. Moreover, it is proved that, under mild additional conditions, the attractive stationary points of the SAEM algorithm correspond to the local maxima of the function. presented to support our findings.

795 citations


Journal ArticleDOI
TL;DR: This paper demonstrates the application of genetic algorithms (GAs) in array pattern synthesis by presenting three examples: two for linear arrays and one involving linear and planar arrays.
Abstract: This paper demonstrates the application of genetic algorithms (GAs) in array pattern synthesis. GAs have the ability to escape from local minima and maxima and are ideally suited for problems where the number of variables is very high. We present three examples: two for linear arrays and one involving linear and planar arrays.

381 citations


Journal ArticleDOI
TL;DR: In this paper, the authors developed a procedure for fitting experimental and simulated X-ray reflectivity and diffraction data in order to automate and quantify the characterization of thin-film structures.
Abstract: We have developed a procedure for fitting experimental and simulated X–ray reflectivity and diffraction data in order to automate and to quantify the characterization of thin–film structures. The optimization method employed is a type of genetic algorithm called‘differential evolution’. The method is capable of rapid convergence to the global minimum of an error function in parameter space even when there are many local minima in addition to the global minimum. We show how to estimate the pointwise errors of the optimized parameters, and how to determine whether the model adequately represents the structure. The procedure is capable of fitting some tens of adjustable parameters, given suitable data.

285 citations


Journal ArticleDOI
TL;DR: The hybrid inversion is found to be faster by more than an order of magnitude for a benchmark testcase in which the form of the geoacoustic model is known and an underparameterized approach is employed to determine a minimum-structure solution.
Abstract: In this paper, local, global, and hybrid inversion algorithms are developed and applied to the problem of determining geoacoustic properties by minimizing the mismatch between measured and modeled acoustic fields. Local inversion methods are sensitive to gradients in the mismatch and move effectively downhill, but generally become trapped in local minima and must be initiated from a large number of starting points. Global inversion methods use a directed random process to search the parameter space for the optimal solution. They include the ability to escape from local minima, but as no gradient information is used, the search can be relatively inefficient. Hybrid inversion methods combine local and global approaches to produce a more efficient and effective algorithm. Here, downhill simplex (local) and simulated annealing (global) methods are developed individually and combined to produce a hybrid simplex simulated annealing algorithm. The hybrid inversion is found to be faster by more than an order of magnitude for a benchmark testcase in which the form of the geoacoustic model is known. The hybrid inversion algorithm is also applied to a testcase consisting of an unknown number of layers representing a general geoacoustic profile. Since the form of the model is not known, an underparameterized approach is employed to determine a minimum-structure solution.

121 citations


Proceedings ArticleDOI
01 Jan 1999
TL;DR: Imposing chirality constraints to limit the search for the plane at infinity to a 3-dimensional cubic region of parameter space is imposed and it is shown that this dense search allows one to avoid areas of local minima effectively and find global minima of the cost function.
Abstract: This paper considers the problem of self-calibration of a camera from an image sequence in the case where the camera's internal parameters (most notably focal length) may change The problem of camera self-calibration from a sequence of images has proven to be a difficult one in practice, due to the need ultimately to resort to non-linear methods, which have often proven to be unreliable In a stratified approach to self-calibration, a projective reconstruction is obtained first and this is successively refined first to an affine and then to a Euclidean (or metric) reconstruction It has been observed that the difficult step is to obtain the affine reconstruction, or equivalently to locate the plane at infinity in the projective coordinate frame The problem is inherently non-linear and requires iterative methods that risk not finding the optimal solution The present paper overcomes this difficulty by imposing chirality constraints to limit the search for the plane at infinity to a 3-dimensional cubic region of parameter space It is then possible to carry out a dense search over this cube in reasonable time For each hypothesised placement of the plane at infinity, the calibration problem is reduced to one of calibration of a nontranslating camera, for which fast non-iterative algorithms exist A cost function based on the result of the trial calibration is used to determine the best placement of the plane at infinity Because of the simplicity of each trial, speeds of over 10,000 trials per second are achieved on a 256 MHz processor It is shown that this dense search allows one to avoid areas of local minima effectively and find global minima of the cost function

119 citations


Journal ArticleDOI
TL;DR: In this paper, a common method is developed to derive efficient GNC-algorithms for the minimization of MAP energies which arise in the context of any observation system giving rise to a convex data-fidelity term and of Markov random field energies involving any nonconvex and/or nonsmooth PFs.
Abstract: This paper is concerned with the reconstruction of images (or signals) from incomplete, noisy data, obtained at the output of an observation system. The solution is defined in maximum a posteriori (MAP) sense and it appears as the global minimum of an energy function joining a convex data-fidelity term and a Markovian prior energy. The sought images are composed of nearly homogeneous zones separated by edges and the prior term accounts for this knowledge. This term combines general nonconvex potential functions (PFs) which are applied to the differences between neighboring pixels. The resultant MAP energy generally exhibits numerous local minima. Calculating its local minimum, placed in the vicinity of the maximum likelihood estimate, is inexpensive but the resultant estimate is usually disappointing. Optimization using simulated annealing is practical only in restricted situations. Several deterministic suboptimal techniques approach the global minimum of special MAP energies, employed in the field of image denoising, at a reasonable numerical cost. The latter techniques are not directly applicable to general observation systems, nor to general Markovian prior energies. This work is devoted to the generalization of one of them, the graduated nonconvexity (GNC) algorithm, in order to calculate nearly-optimal MAP solutions in a wide range of situations. In fact, GNC provides a solution by tracking a set of minima along a sequence of approximate energies, starting from a convex energy and progressing toward the original energy. In this paper, we develop a common method to derive efficient GNC-algorithms for the minimization of MAP energies which arise in the context of any observation system giving rise to a convex data-fidelity term and of Markov random field (MRF) energies involving any nonconvex and/or nonsmooth PFs. As a side-result, we propose how to construct pertinent initializations which allow us to obtain meaningful solutions using local minimization of these MAP energies. Two numerical experiments-an image deblurring and an emission tomography reconstruction-illustrate the performance of the proposed technique.

110 citations


Journal ArticleDOI
TL;DR: An objective function is derived from basic model assumptions which includes the normal tissue constraints as interior penalty functions which leads to constraints similar to dose-volume constraints for organs that are composed of parallel subunits.
Abstract: The implementation of biological optimization of radiation treatment plans is impeded by both computational and modelling problems. We derive an objective function from basic model assumptions which includes the normal tissue constraints as interior penalty functions. For organs that are composed of parallel subunits, a mean response model is proposed which leads to constraints similar to dose-volume constraints. This objective function is convex in the case when no parallel organs lie in the treatment volume. Otherwise, an argument is given to show that a number of local minima may exist which are near degenerate to the global minimum. Thus, together with the measure quality of the objective function, highly efficient gradient algorithms can be used. The number of essential biological model parameters could be reduced to a minimum. However, if the optimization constraints are given as TCP/NTCP values, Lagrange multiplier updates have to be performed by invoking comprehensive biological models.

108 citations


Journal ArticleDOI
TL;DR: An efficient algorithm is derived for topographic mapping of proximity data (TMP), which can be seen as an extension of Kohonen's self-organizing map to arbitrary distance measures and is used to generate a connection map of areas of the cat's cerebral cortex.
Abstract: We derive an efficient algorithm for topographic mapping of proximity data (TMP), which can be seen as an extension of Kohonen's self-organizing map to arbitrary distance measures. The TMP cost function is derived in a Baysian framework of folded Markov chains for the description of autoencoders. It incorporates the data by a dissimilarity matrix and the topographic neighborhood by a matrix of transition probabilities. From the principle of maximum entropy, a nonfactorizing Gibbs distribution is obtained, which is approximated in a mean-field fashion. This allows for maximum likelihood estimation using an expectation-maximization algorithm. In analogy to the transition from topographic vector quantization to the self-organizing map, we suggest an approximation to TMP that is computationally more efficient. In order to prevent convergence to local minima, an annealing scheme in the temperature parameter is introduced, for which the critical temperature of the first phase transition is calculated in terms o...

94 citations


Book ChapterDOI
TL;DR: This paper presents an interpretation of a classic optical flow method by Nagel and Enkelmann as a tensor-driven anisotropic diffusion approach in digital image analysis that can recover displacement fields which are far beyond the typical one-pixel limits that are characteristic for many Differential methods for optical flow recovery.
Abstract: This paper presents an interpretation of a classic optical flow method by Nagel and Enkelmann as a tensor-driven anisotropic diffusion approach in digital image analysis. We introduce an improvement into the model formulation, and we establish well-posedness results for the resulting system of parabolic partial Differential equations. Our method avoids linearizations in the optical flow constraint, and it can recover displacement fields which are far beyond the typical one-pixel limits that are characteristic for many Differential methods for optical flow recovery. A robust numerical scheme is presented in detail. We avoid convergence to irrelevant local minima by embedding our method into a linear scale-space framework and using a focusing strategy from coarse to fine scales. The high accuracy of the proposed method is demonstrated by means of a synthetic and a real-world image sequence.

90 citations


Journal ArticleDOI
TL;DR: In this article, a variant of a traditional genetic algorithm, known as a niching genetic algorithm (NGA), is presented for solving the inversion of teleseismic body waves for the source parameters of the M W 7.2 Kuril Islands event of 2 February 1996.
Abstract: We present a variant of a traditional genetic algorithm, known as a niching genetic algorithm (NGA), which is effective at multimodal function optimization. Such an algorithm is useful for geophysical inverse problems that contain more than one distinct solution. We illustrate the utility of an NGA via a multimodal seismological inverse problem: the inversion of teleseismic body waves for the source parameters of the M W 7.2 Kuril Islands event of 2 February 1996. We assume the source to be a pure double-couple event and so parametrize our models in terms of strike, dip, and slip, guaranteeing that two global minima exist, one of which represents the fault plane and the other the auxiliary plane. We use ray theory to compute the fundamental P and SH synthetic seismograms for a given source-receiver geometry; the synthetics for an arbitrary fault orientation are produced by taking linear combinations of these fundamentals, yielding a computationally fast forward problem. The NGA is successful at determining that two major solutions exist and at maintaining the solutions in a steady state. Several inferior solutions representing local minima of the objective function are found as well. The two best focal solutions we find for the Kuril Islands event are very nearly conjugate planes and are consistent with the focal planes reported by the Harvard CMT project. The solutions indicate thrust movement on a moderately dipping fault—a source typical of the convergent margin near the Kuril Islands.

80 citations


Book ChapterDOI
11 Oct 1999
TL;DR: A new vector-based definition of descent directions in discrete space is proposed and it is shown that the new definition does not obey the rules of calculus in continuous space, but provides a strong mathematical foundation for solving general nonlinear discrete optimization problems.
Abstract: In this paper we present a Lagrange-multiplier formulation of discrete constrained optimization problems, the associated discrete-space first-order necessary and sufficient conditions for saddle points, and an efficient first-order search procedure that looks for saddle points in discrete space. Our new theory provides a strong mathematical foundation for solving general nonlinear discrete optimization problems. Specifically, we propose a new vector-based definition of descent directions in discrete space and show that the new definition does not obey the rules of calculus in continuous space. Starting from the concept of saddle points and using only vector calculus, we then prove the discrete-space first-order necessary and sufficient conditions for saddle points. Using well-defined transformations on the constraint functions, we further prove that the set of discrete-space saddle points is the same as the set of constrained local minima, leading to the first-order necessary and sufficient conditions for constrained local minima. Based on the first-order conditions, we propose a local-search method to look for saddle points that satisfy the first-order conditions.

Journal ArticleDOI
TL;DR: The problem of identifying weak sharp minima of order m, an important class of (possibly) nonisolated minima, is investigated, and some characterizations of weak sharp minimality are obtained.
Abstract: The problem of identifying weak sharp minima of order m, an important class of (possibly) nonisolated minima, is investigated in this paper. Some sufficient conditions for weak sharp minimality in nonsmooth mathematical programming are presented, and some characterizations of weak sharp minimality are obtained, with special attention given to orders one and two. It is also demonstrated that two of these sufficient conditions guarantee exactness of an l1 penalty function. A key role in this paper is played by two geometric concepts: the limiting proximal normal cone and a generalization of the contingent cone.

Journal ArticleDOI
TL;DR: This paper analyzes and compares the well-known gradient descent algorithm and the more recent exponentiated gradient algorithm for training a single neuron with an arbitrary transfer function and proves worst-case loss bounds for both algorithms in the single neuron case.
Abstract: We analyze and compare the well-known gradient descent algorithm and the more recent exponentiated gradient algorithm for training a single neuron with an arbitrary transfer function. Both algorithms are easily generalized to larger neural networks, and the generalization of gradient descent is the standard backpropagation algorithm. We prove worst-case loss bounds for both algorithms in the single neuron case. Since local minima make it difficult to prove worst case bounds for gradient-based algorithms, we must use a loss function that prevents the formation of spurious local minima. We define such a matching loss function for any strictly increasing differentiable transfer function and prove worst-case loss bounds for any such transfer function and its corresponding matching loss. The different forms of the two algorithms' bounds indicates that exponentiated gradient outperforms gradient descent when the inputs contain a large number of irrelevant components. Simulations on synthetic data confirm these analytical results.

Journal ArticleDOI
TL;DR: In this article, a special variable metric method is given for finding minima of convex functions that are not necessarily differentiable, and some encouraging numerical experience is reported for global convergence of the method.
Abstract: A special variable metric method is given for finding minima of convex functions that are not necessarily differentiable. Time-consuming quadratic programming subproblems do not need to be solved. Global convergence of the method is established. Some encouraging numerical experience is reported.

Book ChapterDOI
11 Oct 1999
TL;DR: Con constrained simulated annealing (CSA), a global minimization algorithm that converges to constrained global minima with probability one, is presented, for solving nonlinear discrete non-convex constrained minimization problems.
Abstract: In this paper, we present constrained simulated annealing (CSA), a global minimization algorithm that converges to constrained global minima with probability one, for solving nonlinear discrete non-convex constrained minimization problems. The algorithm is based on the necessary and sufficient condition for constrained local minima in the theory of discrete Lagrange multipliers we developed earlier. The condition states that the set of discrete saddle points is the same as the set of constrained local minima when all constraint functions are non-negative. To find the discrete saddle point with the minimum objective value, we model the search by a finite inhomogeneous Markov chain that carries out (in an annealing fashion) both probabilistic descents of the discrete Lagrangian function in the original-variable space and probabilistic ascents in the Lagrange-multiplier space. We then prove the asymptotic convergence of the algorithm to constrained global minima with probability one. Finally, we extend CSA to solve nonlinear constrained problems with continuous variables and those with mixed (both discrete and continuous) variables. Our results on a set of nonlinear benchmarks are much better than those reported by others. By achieving asymptotic convergence, CSA is one of the major developments in nonlinear constrained global optimization today.

Journal ArticleDOI
TL;DR: A reformulation of the usual least-squares waveform inversion problem is proposed to retrieve, from seismic data, the kinematic parameters (the two-dimensional background velocity and the source and cable depth) by local optimization by using a local optimization technique.
Abstract: A reformulation of the usual least-squares waveform inversion problem is proposed to retrieve, from seismic data, the kinematic parameters (the two-dimensional (2D) background velocity and the source and cable depth) by local optimization. These last parameters are of paramount importance for a successful inversion of very high resolution (VHR) seismic data which we are interested in. In our inversion the source and cable depth parameters are treated in the same way as the background velocity. To avoid the problem of local minima, a change of unknowns is performed: the depth reflectivity is replaced by its dual variable, called the time reflectivity. In this way, the current value of the reflectivity is stored in the time domain and is strongly decoupled from the current value of the velocity field and cable depth. The increase in modeling complexity (an additional prestack migration is required for each function evaluation) is compensated by the enlargement of the attraction domain of the global minimum which allows the use of a local optimization technique. A numerical implementation with Born approximation and ray tracing is detailed, in which the derivatives of the travel time are computed via an adjoint state technique for more efficiency. Numerical results illustrate the behavior of the new objective function, and inversion of synthetic and real VHR data for kinematic parameters is performed.

Journal ArticleDOI
TL;DR: A bootstrap-based method for enhancing a search through a space of models is proposed, well suited to complex, adaptively fitted models and provides a convenient method for finding better local minima and for resistant fitting.
Abstract: We propose a bootstrap-based method for enhancing a search through a space of models. The technique is well suited to complex, adaptively fitted models—it provides a convenient method for finding better local minima and for resistant fitting. Applications to regression, classification, and density estimation are described. We also provide results on the asymptotic behavior of bumping estimates.

Journal Article
TL;DR: An ESS is defined as a set of strategies that is both resistant to invasion and convergent-stable, and how the number of co-existing strategies emerges when seeking an ESS solution is demonstrated.
Abstract: We use a fitness-generating function (G-function) approach to evolutionary games. The Gfunction allows for simultaneous consideration of strategy dynamics and population dynamics. In contrast to approaches using a separate fitness function for each strategy, the G-function automatically expands and contracts the dimensionality of the evolutionary game as the number of extant strategies increases or decreases. In this way, the number of strategies is not fixed but emerges as part of the evolutionary process. We use the G-function to derive conditions for a strategy’s (or a set of strategies) resistance to invasion and convergence stability. In hopes of relating the proliferation of ESS-related terminology, we define an ESS as a set of strategies that is both resistant to invasion and convergent-stable. With our definition of ESS, we show the following: (1) Evolutionarily unstable maxima and minima are not achievable from adaptive dynamics. (2) Evolutionarily stable minima are achievable from adaptive dynamics and allow for adaptive speciation and divergence by additional strategies – in this sense, these minima provide transition points during an adaptive radiation and are therefore unstable when subject to small mutations. (3) Evolutionarily stable maxima are both invasion-resistant and convergent-stable. When global maxima on the adaptive landscape are at zero fitness, these combinations of strategies make up the ESS. We demonstrate how the number of co-existing strategies (coalition) emerges when seeking an ESS solution. The Lotka-Volterra competition model and Monod model of competition are used to illustrate combinations of invasion resistance and convergence stability, adaptive speciation and evolutionarily ‘stable’ minima, and the diversity of co-existing strategies that can emerge as the ESS.

Patent
03 Jun 1999
TL;DR: In this paper, a recombination operator combines two or more parameter vectors from one iteration of simulated scattering data to form a new parameter vector for the next iteration, in a manner such that there is a high probability that the new parameter will better fit the experimental data than any of the parent parameters.
Abstract: Evolutionary algorithms are used to find a global solution to the fitting of experimental X-ray scattering data to simulated models. A recombination operator combines two or more parameter vectors from one iteration of simulated scattering data to form a new parameter vector for the next iteration, in a manner such that there is a high probability that the new parameter will better fit the experimental data than any of the parent parameters. A mutation operator perturbs the value of a parent vector, to permit new regions of the error function to be examined, and thereby avoid settling on local minima. The natural selection guarantees that the parameter vectors with the best fitness will be propagated into future iterations.

Journal ArticleDOI
TL;DR: A novel heuristics approach for neural networks global learning algorithm based upon the least-squares method and a Penalty approach to solve the problem of local minima that outperforms other conventional algorithms in terms of convergence speed and the ability of escaping from theLocal minima.

Journal ArticleDOI
TL;DR: In this paper, the convergence of the generalized simulated annealing with time-inhomogeneous communication cost functions is discussed, based on the use of log-Sobolev inequalities and semigroup techniques.
Abstract: The convergence of the generalized simulated annealing with time-inhomogeneous communication cost functions is discussed. This study is based on the use of log-Sobolev inequalities and semigroup techniques in the spirit of a previous article by one of the authors. We also propose a natural test set approach to study the global minima of the virtual energy. The second part of the paper is devoted to the application of these results. We propose two general Markovian models of genetic algorithms and we give a simple proof of the convergence toward the global minima of the fitness function. Finally we introduce a stochastic algorithm that converges to the set of the global minima of a given mean cost optimization problem.

Patent
David Barral1
08 Oct 1999
TL;DR: In this paper, a modified simulated annealing method is applied to the problem of robot placement in CAD systems, in the context of welding tasks, in order to save CPU time.
Abstract: A method and system for optimizing the placement of a robot in a workplace so as to minimize cycle time is defined. A modified simulated annealing method (SA) is applied to the problem of robot placement in CAD systems, in the context of welding tasks. The objective function for optimization is cycle time, which can be obtained from available robotic CAD software. The research domains are simplified, and the SA method is applied to yield an optimal or near-optimal solution to each problem. To obtain the optimal placement of the robot, the research domain is first simplified by determining an acceptable base location domain, then obstacle shadows, which are subtracted from the previous domain to give the free acceptable base location domain. A modified SA method is applied to this domain, using task feasibility tests before simulating the cycle time, in order to save CPU time. The modified SA technique gives a set of near-optimal placements, together with other local minima, and an estimate of an efficient region where the robot may be positioned.

Journal ArticleDOI
TL;DR: This work revisits the oft-studied asymptotic behavior of the parameter or weight estimate returned by any member of a large family of neural network training algorithms and rigorously establishes conditions under which the parameter estimate converges strongly into the set of minima of the generalization error.
Abstract: We revisit the oft-studied asymptotic (in sample size) behavior of the parameter or weight estimate returned by any member of a large family of neural network training algorithms. By properly accounting for the characteristic property of neural networks that their empirical and generalization errors possess multiple minima, we rigorously establish conditions under which the parameter estimate converges strongly into the set of minima of the generalization error. Convergence of the parameter estimate to a particular value cannot be guaranteed under our assumptions. We then evaluate the asymptotic distribution of the distance between the parameter estimate and its nearest neighbor among the set of minima of the generalization error. Results on this question have appeared numerous times and generally assert asymptotic normality, the conclusion expected from familiar statistical arguments concerned with maximum likelihood estimators. These conclusions are usually reached on the basis of somewhat informal calculations, although we shall see that the situation is somewhat delicate. The preceding results then provide a derivation of learning curves for generalization and empirical errors that leads to bounds on rates of convergence.

Proceedings ArticleDOI
01 Aug 1999
TL;DR: For the optimizations required by the controller synthesis and state estimation of MLD systems, the proposed algorithm reduces the average number of node explorations during the search of a global minimum and provides good local minima after a short number of steps of the Branch and Bound procedure.
Abstract: This paper presents a new Branch and Bound tree exploring strategy for solving Mixed Integer Quadratic Programs (MIQP)involving time evolutions of linear hybrid systems. In particular, we refer to the Mixed Logical Dynamical (MLD) models introduced by Bemporad and Morari (1999), where the hybrid system is described by linear equations/inequalities involving continuous and integer variables. For the optimizations required by the controller synthesis and state estimation of MLD systems, the proposed algorithm reduces the average number of node explorations during the search of a global minimum. It also provides good local minima after a short number of steps of the Branch and Bound procedure.

Journal ArticleDOI
TL;DR: This study examines robustness of master equations based only on statistical samples of the surface topography, to make it possible to work with larger systems for which a full topographical description is either impossible or infeasible.
Abstract: Prior work [K. D. Ball and R. S. Berry, J. Chem. Phys. 109, 8541 (1998); 109, 8557 (1998)] has demonstrated that master equations constructed from a complete set of minima and transition states can capture the essential features of the relaxation dynamics of small systems. The current study extends this work by examining robustness of master equations based only on statistical samples of the surface topography, to make it possible to work with larger systems for which a full topographical description is either impossible or infeasible. We ask whether such “statistical” master equations can predict relaxation on the entire potential energy surface. Our test cases are Ar11 and Ar13, for which we have extensive databases: 168 geometrically distinct minima and 1890 transition states for Ar11, and 1478 minima and 17,357 saddles for Ar13 which we assume represent complete set of stationary points. From these databases we construct statistical sample sets of transition sequences, and compare relaxation predictio...

Proceedings Article
31 Jul 1999
TL;DR: A new framework for extending consistent domains of numeric CSP is introduced which provides an efficient and incremental algorithm which computes the maximal extension of the domain of one variable.
Abstract: This paper introduces a new framework for extending consistent domains of numeric CSP. The aim is to offer the greatest possible freedom of choice for one variable to the designer of a CAD application. Thus, we provide here an efficient and incremental algorithm which computes the maximal extension of the domain of one variable. The key point of this framework is the definition, for each inequality, of an univariate extrema function which computes the left most and right most solutions of a selected variable (in a space delimited by the domains of the other variables). We show how these univariate extrema functions can be implemented efficiently. The capabilities of this approach are illustrated on a ballistic example.

Journal ArticleDOI
TL;DR: It turns out that simulated annealing does not only avoid local minima, but also helps to give a clearer picture of the Edgeworth-Pareto optimal set and improves the judgement of the influence of nonsensitive parameters.
Abstract: Optimization methods became an important tool for the synthesis of complex mechanical systems. However, while deterministic approaches usually yield good convergence within a few iterations, they often lead to local minima only. Stochastic optimization, like simulated annealing, is not as sensitive with respect to local minima, but often requires high computational effort. In this paper, we describe an optimization concept founded on either a deterministic gradient-based method or a stochastic simulated annealing optimization procedure. An application to vehicle dynamics and a comparison of the different procedures is given. It turns out that simulated annealing does not only avoid local minima, but also helps to give a clearer picture of the Edgeworth-Pareto optimal set and improves the judgement of the influence of nonsensitive parameters. Beside efficiency, these properties also have to be considered when it is decided whether a stochastic or a deterministic optimization algorithm should be chosen.

Journal ArticleDOI
TL;DR: In this article, the authors present a unified discussion of support information and zero locations in the reconstruction of a discrete complex image from Fourier-transform phaseless data, and demonstrate robustness against false solutions, starting from completely random first guesses.
Abstract: A recently introduced approach to phase-retrieval problems is applied to present a unified discussion of support information and zero locations in the reconstruction of a discrete complex image from Fourier-transform phaseless data. The choice of the square-modulus function of the Fourier transform of the unknown as the problem datum results in a quadratic operator that has to be inverted, i.e., a simple nonlinearity. This circumstance makes it possible to consider and to point out some relevant factors that affect the local minima problem that arises in the solution procedure (which amounts to minimizing a quartic functional). Simple modifications of the basic procedure help to explain the role of support information and zeros in the data and to develop suitable strategies for avoiding the local minima problem. All results can be summarized by reference to the ratio between the effective dimensions of the data space and the space of unknowns. Numerical results identify the approach’s considerable robustness against false solutions, starting from completely random first guesses, if the above ratio is larger than 3. The algorithm also ensures robust performance in the presence of noise in the data.

Proceedings ArticleDOI
08 Nov 1999
TL;DR: This paper improves constrained simulated annealing (CSA), a discrete global minimization algorithm with asymptotic convergence to discrete constrained global minima with probability one, and extends CSA to solve nonlinear continuous constrained optimization problems whose variables take continuous values.
Abstract: This paper improves constrained simulated annealing (CSA), a discrete global minimization algorithm with asymptotic convergence to discrete constrained global minima with probability one. The algorithm is based on the necessary and sufficient conditions for discrete constrained local minima in the theory of discrete Lagrange multipliers. We extend CSA to solve nonlinear continuous constrained optimization problems whose variables take continuous values. We evaluate many heuristics, such as dynamic neighborhoods, gradual resolution of nonlinear equality constraints and reannealing, in order to greatly improve the efficiency of solving continuous problems. We report much better solutions than the best-known solutions in the literature on two sets of continuous optimization benchmarks.

Journal ArticleDOI
TL;DR: The new back-propagation algorithm is to change the derivative of the activation function so as to magnify the backward propagated error signal, thus the convergence rate can be accelerated and the local minimum can be escaped.
Abstract: The conventional back-propagation algorithm is basically a gradient-descent method, it has the problems of local minima and slow convergence. A new generalized back-propagation algorithm which can effectively speed up the convergence rate and reduce the chance of being trapped in local minima is introduced. The new back-propagation algorithm is to change the derivative of the activation function so as to magnify the backward propagated error signal, thus the convergence rate can be accelerated and the local minimum can be escaped. In this letter, we also investigate the convergence of the generalized back-propagation algorithm with constant learning rate. The weight sequences in generalized back-propagation algorithm can be approximated by a certain ordinary differential equation (ODE). When the learning rate tends to zero, the interpolated weight sequences of generalized back-propagation converge weakly to the solution of associated ODE.