Showing papers on "Maxima and minima published in 2000"

Reliable Estimation of Dense Optical Flow Fields with Large Displacements

Abstract: In this paper we show that a classic optical flow technique by Nagel and Enkelmann (1986, IEEE Trans. Pattern Anal. Mach. Intell., Vol. 8, pp. 565–593) can be regarded as an early anisotropic diffusion method with a diffusion tensor. We introduce three improvements into the model formulation that (i) avoid inconsistencies caused by centering the brightness term and the smoothness term in different images, (ii) use a linear scale-space focusing strategy from coarse to fine scales for avoiding convergence to physically irrelevant local minima, and (iii) create an energy functional that is invariant under linear brightness changes. Applying a gradient descent method to the resulting energy functional leads to a system of diffusion–reaction equations. We prove that this system has a unique solution under realistic assumptions on the initial data, and we present an efficient linear implicit numerical scheme in detail. Our method creates flow fields with 100 % density over the entire image domain, it is robust under a large range of parameter variations, and it can recover displacement fields that are far beyond the typical one-pixel limits which are characteristic for many differential methods for determining optical flow. We show that it performs better than the optical flow methods with 100 % density that are evaluated by Barron et al. (1994, Int. J. Comput. Vision, Vol. 12, pp. 43–47). Our software is available from the Internet.

The heavy ball with friction method, i. the continuous dynamical system: global exploration of the local minima of a real-valued function by asymptotic analysis of a dissipative dynamical system

Abstract: Let H be a real Hilbert space and Φ:H ↦ R a continuously differentiable function, whose gradient is Lipschitz continuous on bounded sets. We study the nonlinear dissipative dynamical system: ${\ddot x}(t)+\lambda{\dot x}(t)+ abla\Phi(x(t))=0, \lambda>0, plus Cauchy data, mainly in view of the unconstrained minimization of the function Φ. New results concerning the convergence of a solution to a critical point are given in various situations, including when Φ is convex (possibly with multiple minima) or is a Morse function (the critical point being then generically a local minimum); a counterexample shows that, without peculiar assumptions, a trajectory may not converge. By following the trajectories, we obtain a method for exploring local minima of Φ. A singular perturbation analysis links our results with those concerning gradient systems.

Global Optimization on Funneling Landscapes

Abstract: Molecular conformation problems arising in computational chemistry require the global minimization of a non-convex potential energy function representing the interactions of, for example, the component atoms in a molecular system. Typically the number of local minima on the potential energy surface grows exponentially with system size, and often becomes enormous even for relatively modestly sized systems. Thus the simple multistart strategy of randomly sampling local minima becomes impractical. However, for many molecular conformation potential energy surfaces the local minima can be organized by a simple adjacency relation into a single or at most a small number of funnels. A distinguished local minimum lies at the bottom of each funnel and a monotonically descending sequence of adjacent local minima connects every local minimum in the funnel with the funnel bottom. Thus the global minimum can be found among the comparatively small number of funnel bottoms, and a multistart strategy based on sampling funnel bottoms becomes viable. In this paper we present such an algorithm of the basin-hopping type and apply it to the Lennard–Jones cluster problem, an intensely studied molecular conformation problem which has become a benchmark for global optimization algorithms. Results of numerical experiments are presented which confirm both the multifunneling character of the Lennard–Jones potential surface as well as the efficiency of the algorithm. The algorithm has found all of the current putative global minima in the literature up to 110 atoms, as well as discovered a new global minimum for the 98-atom cluster of a novel geometrical class.

Coupled Geodesic Active Regions for Image Segmentation: A Level Set Approach

Abstract: This paper presents a novel variational method for image segmentation that unifies boundary and region-based information sources under the Geodesic Active Region framework. A statistical analysis based on the Minimum Description Length criterion and the Maximum Likelihood Principle for the observed density function (image histogram) using a mixture of Gaussian elements, indicates the number of the different regions and their intensity properties. Then, the boundary information is determined using a probabilistic edge detector, while the region information is estimated using the Gaussian components of the mixture model. The defined objective function is minimized using a gradient-descent method where a level set approach is used to implement the resulting PDE system. According to the motion equations, the set of initial curves is propagated toward the segmentation result under the influence of boundary and region-based segmentation forces, and being constrained by a regularity force. The changes of topology are naturally handled thanks to the level set implementation, while a coupled multi-phase propagation is adopted that increases the robustness and the convergence rate by imposing the idea of mutually exclusive propagating curves. Finally, to reduce the required computational cost and the risk of convergence to local minima, a multi-scale approach is also considered. The performance of our method is demonstrated on a variety of real images.

Global Optimization for the Parameter Estimation of Differential-Algebraic Systems

Abstract: The estimation of parameters in semiempirical models is essential in numerous areas of engineering and applied science. In many cases these models are represented by a set of nonlinear differential-algebraic equations. This introduces difficulties from both a numerical and an optimization perspective. One such difficulty, which has not been adequately addressed, is the existence of multiple local minima. In this paper, two novel global optimization methods will be presented which offer a theoretical guarantee of convergence to the global minimum for a wide range of problems. The first is based on converting the dynamic system of equations into a set of algebraic constraints through the use of collocation methods. The reformulated problem has interesting mathematical properties which allow for the development of a deterministic branch and bound global optimization approach. The second method is based on the use of integration to solve the dynamic system of equations. Both methods will be applied to the pro...

Global Optimization of Costly Nonconvex Functions Using Radial Basis Functions

Abstract: The paper considers global optimization of costly objective functions, i.e. the problem of finding the global minimum when there are several local minima and each function value takes considerable CPU time to compute. Such problems often arise in industrial and financial applications, where a function value could be a result of a time-consuming computer simulation or optimization. Derivatives are most often hard to obtain, and the algorithms presented make no use of such information. Several algorithms to handle the global optimization problem are described, but the emphasis is on a new method by Gutmann and Powell, A radial basis function method for global optimization. This method is a response surface method, similar to the Efficient Global Optimization (EGO) method of Jones. Our Matlab implementation of the Radial Basis Function (RBF) method is described in detail and we analyze its efficiency on the standard test problem set of Dixon-Szego, as well as its applicability on a real life industrial problem from train design optimization. The results show that our implementation of the RBF algorithm is very efficient on the standard test problems compared to other known solvers, but even more interesting, it performs extremely well on the train design optimization problem.

Deterministic Global Optimization in Nonlinear Optimal Control Problems

Abstract: The accurate solution of optimal control problems is crucial in many areas of engineering and applied science. For systems which are described by a nonlinear set of differential-algebraic equations, these problems have been shown to often contain multiple local minima. Methods exist which attempt to determine the global solution of these formulations. These algorithms are stochastic in nature and can still get trapped in local minima. There is currently no deterministic method which can solve, to global optimality, the nonlinear optimal control problem. In this paper a deterministic global optimization approach based on a branch and bound framework is introduced to address the nonlinear optimal control problem to global optimality. Only mild conditions on the differentiability of the dynamic system are required. The implementa-tion of the approach is discussed and computational studies are presented for four control problems which exhibit multiple local minima.

A genetic algorithm for the structural optimization of Morse clusters

Abstract: This article describes the application of a genetic algorithm for the structural optimization of 19–50-atom clusters bound by medium-range and short-range Morse pair potentials. The GA is found to be efficient and reliable for finding the geometries corresponding to the previously published global minima [Doye JPK, Wales DJ (1997) J Chem Soc Faraday Trans 93: 4233]. Using the genetic algorithm, only a relatively small number of energy evaluations and minimizations are required to find the global minima. By contrast, a simple random search algorithm often cannot find the global minima of the larger clusters, even after many thousands of searches.

Human-Guided Simple Search

Abstract: Scheduling, routing, and layout tasks are examples of hard operations-research problems that have broad application in industry. Typical algorithms for these problems combine some form of gradient descent to find local minima with some strategy for escaping nonoptimal local minima. Our idea is to divide these two subtasks cleanly between human and computer: in our paradigm of human-guided simple search the computer is responsible only for finding local minima using a simple hill-climbing search; using visualization and interaction techniques, the human user identifies promising regions of the search space for the computer to explore, and intervenes to help it escape nonoptimal local minima. We have applied our approach to the problem of capacitated vehicle routing with time windows, a commercially important problem with a rich research history. Despite its simplicity, our prototype system is competitive with the majority of previously reported systems on benchmark academic problems, and has the advantage of keeping a human tightly in the loop to handle the complexities of real-world applications. AAAI 2000

Registration of range data using a hybrid simulated annealing and iterative closest point algorithm

Abstract: The need to register data is abundant in applications such as: world modeling, part inspection and manufacturing, object recognition, pose estimation, robotic navigation, and reverse engineering Registration occurs by aligning the regions that are common to multiple images The largest difficulty in performing this registration is dealing with outliers and local minima while remaining efficient A commonly used technique, iterative closest point, is efficient but is unable to deal with outliers or avoid local minima Another commonly used optimization algorithm, simulated annealing, is effective at dealing with local minima but is very slow Therefore, the algorithm developed in this paper is a hybrid algorithm that combines the speed of iterative closest point with the robustness of simulated annealing Additionally, a robust error function is incorporated to deal with outliers This algorithm is incorporated into a complete modeling system that inputs two sets of range data, registers the sets, and outputs a composite model

Optimal Structure from Motion: Local Ambiguities and Global Estimates

Abstract: “Structure From Motion” (SFM) refers to the problem of estimating spatial properties of a three-dimensional scene from the motion of its projection onto a two-dimensional surface, such as the retina. We present an analysis of SFM which results in algorithms that are provably convergent and provably optimal with respect to a chosen norm. In particular, we cast SFM as the minimization of a high-dimensional quadratic cost function, and show how it is possible to reduce it to the minimization of a two-dimensional function whose stationary points are in one-to-one correspondence with those of the original cost function. As a consequence, we can plot the reduced cost function and characterize the configurations of structure and motion that result in local minima. As an example, we discuss two local minima that are associated with well-known visual illusions. Knowledge of the topology of the residual in the presence of such local minima allows us to formulate minimization algorithms that, in addition to provably converge to stationary points of the original cost function, can switch between different local extrema in order to converge to the global minimum, under suitable conditions. We also offer an experimental study of the distribution of the estimation error in the presence of noise in the measurements, and characterize the sensitivity of the algorithm using the structure of Fisher's Information matrix.

Hoffman's Error Bound, Local Controllability, and Sensitivity Analysis

Abstract: Our aim is to present sufficient conditions ensuring Hoffman's error bound for lower semicontinuous nonconvex inequality systems and to analyze its impact on the local controllability, implicit function theorem for (non-Lipschitz) multivalued mappings, generalized equations (variational inequalities), and sensitivity analysis and on other problems like Lipschitzian properties of polyhedral multivalued mappings as well as weak sharp minima or linear conditioning. We show how the information about our sufficient conditions can be used to provide a computable constant such that Hoffman's error bound holds. We also show that this error bound is nothing but the classical Farkas lemma for linear inequality systems. In the latter case our constant may be computed explicitly.

Generalized dual space indicator method for underwater imaging

Abstract: This paper presents a generalized dual space indicator method for imaging an obstacle in ocean environments. The method is based on the observation that the combination (weighted integration) of the measured scattered field can approximate the Green function very well when the Green function's source point is inside the obstacle, but not so well when the source is outside the obstacle. We set up an integral equation whose right-hand side is the Green function with a source point from a searching region. From our numerical experiments, we notice that the norm of the solution of the integral equation has local extrema that lie inside the unknown obstacle. Plotting the norm as a function of the source point in the searching region, and filtering out the region with no local extrema of the norm, we obtain a good image of the unknown obstacle. Imaging algorithms and numerical examples are presented.

Localization using Bartlett matched-field processor sidelobes

Abstract: Ambiguity surface sidelobes generated by the Bartlett matched-field processor (MFP) shift location with frequency This sidelobe shift can be viewed as a continuous trajectory in a range-frequency plane at a fixed depth, where the trajectories converge to the correct source range for a perfectly matched surface In isovelocity or bottom-interacting environments the sidelobe trajectories are straight lines that converge to the true range at zero frequency, while environments with upward-refracting sound-speed profiles have trajectories that asymptotically converge as the frequency approaches infinity This behavior can be explained by the theory of waveguide invariants, which predict the local behavior of interference maxima/minima of acoustic intensity in the frequency-range plane As the ambiguity surface of the Bartlett matched-field processor has a physical interpretation in terms of a time-reversed acoustic field, with the sidelobes analogous to local interference maxima, these invariant concepts can be reformulated for application to MFP These interference trajectories are demonstrated to exist in simulations, broadband source tows, and a type A blue whale vocalization Sidelobe trajectories also exist in the range-depth plane, but they contain no information about the correct source depth An appendix demonstrates how these sidelobe properties can be exploited when combining ambiguity surfaces through use of gradient and Radon transform information The resulting range estimators demonstrate better peak-to-sidelobe ratios than a simple incoherent average

Comparison of public-domain software for black box global optimization ∗

Abstract: We instance our experience with six public-domain global optimization software products and report comparative computational results obtained on a set of eleven test problems. The techniques used by the software under study include integral global optimization, genetic algorithms, simulated annealing, clustering, random search, continuation, Bayesian, tunneling, and multi-level methods. The test set contains practical problems: least median of squares regression, protein folding, and multidimensional scaling. These include non-differentiable, and also discontinuous objective functions, some with an exponential number of local minima. The dimension of the search space ranges from 1 to 20. We evaluate the software in view of engineers addressing black box global optimization problems, i.e. problems with an objective function whose explicit form is unknown and whose evaluation is costly. Such an objective function is common in industry. It is for instance given under the form of computer programmes involving...

Improvement of numerical instabilities in topology optimization using the SLP method

Abstract: In this paper, we present a method for preventing numerical instabilities such as checkerboards, mesh-dependencies and local minima occurring in the topology optimization which is formulated by the homogenization design method and in which the SLP method is used as optimizer. In the present method, a function based on the concept of gravity (which we named "the gravity control function") is added to the objective function. The density distribution of the topology is concentrated by maximizing this function, and as a result, checkerboards and intermediate densities are eliminated. Some techniques are introduced in the optimization procedure for preventing the local minima. The validity of the present method is demonstrated by numerical examples of both the short cantilever beam and the MBB beam.

Solving the local minima problem for a mobile robot by classification of spatio-temporal sensory sequences

Abstract: The local minima problem occurs when a robot navigating past obstacles towards a desired target with no priori knowledge of the environment gets trapped in a loop. This happens especially if the environment consists of concave obstacles, mazes, and the like. To come out of the loop the robot must comprehend its repeated traversal through the same environment, which involves memorizing the environment already seen. This paper proposes a new real-time collision avoidance algorithm with the local minima problem solved by classifying the environment based on the spatio-temporal sensory sequences. A double layered classification scheme is adopted. A fuzzy rule base does the spatial classification at the first level and at the second level Kohonen’s self-organizing map and a fuzzy ART network is used for temporal classification. The robot has no prior knowledge of the environment and fuzzy rules govern its obstacle repulsing and target attracting behaviors. As the robot traverses the local environment is modeled and stored in the form of neurons whose weights represent the spatio-temporal sequence of sensor readings. A repetition of a similar environment is mapped to the same neuron in the network and this principle is exploited to identify a local minima situation. Suitable steps are taken to pull the robot out of the local minima. The method has been tested on various complex environments with obstacle loops and mazes, and its efficacy has been established. 2000 John Wiley & Sons, Inc.

Global Optimization for Constrained Nonlinear Programming

Abstract: In this thesis, we develop constrained simulated annealing (CSA), a global optimization algorithm that asymptotically converges to constrained global minima (CGM) with probability one, for solving discrete constrained nonlinear programming problems (NLPs). The algorithm is based on the necessary and sufficient condition for constrained local minima (CLM) in the theory of discrete constrained optimization using Lagrange multipliers developed in our group. The theory proves the equivalence between the set of discrete saddle points and the set of CLM, leading to the first-order necessary and sufficient condition for CLM. To find a CGM, CSA searches for a discrete saddle point with the minimum objective value by carrying out both probabilistic descents in the original-variable space of a discrete augmented Lagrangian function and probabilistic ascents in the Lagrange-multiplier space. We prove that CSA converges asymptotically to a CGM with probability one. We also extend CSA to solve continuous and mixed-integer constrained NLPs. By achieving asymptotic convergence, CSA represents one of the major developments in nonlinear constrained global optimization today, which complements simulated annealing (SA) in unconstrained global optimization. Based on CSA, we have studied various strategies of CSA and their trade-offs for solving continuous, discrete, and mixed-integer NLPs. The strategies evaluated include adaptive neighborhoods, distributions to control sampling, acceptance probabilities, and cooling schedules. An optimization software package based on CSA and its various strategies has been implemented. Finally, we apply CSA to solve a collection of engineering application benchmarks and design filters for subband image coding. Much better results have been reported in comparison with other existing methods.

Response surface model development for global/local optimization using radial basis functions*

Abstract: This paper considers the optimization of complex multi-parameter systems in which the objective function is not known explicitly, and can only be evaluated either through costly physical experiments or through computationally intensive numerical simulation. Furthermore, the objective function of interest may contain many local extrema. Given a data set consisting of the value of the objective function at a scattered set of parameter values, we are interested in developing a response surface model to reduce dramatically the required computation time for parameter optimization runs.To accomplish these tasks, a response surface model is developed using radial basis functions. Radial basis functions provide a way of creating a model whose objective function values match those of the original system at all sampled data points. Interpolation to any other point is easily accomplished and generates a model which represents the system over the entire parameter space. This paper presents the details of the use

Crystal Structure Prediction by Global Optimization as a Tool for Evaluating Potentials: Role of the Dipole Moment Correction Term in Successful Predictions

Abstract: A recently proposed method for surmounting the multiple-minima problem in protein folding is applied here to the prediction of crystal structures by global optimization of a potential energy function. The method, self-consistent basin-to-deformed-basin mapping, locates a group of large basins (regions of attraction of single minima) containing low-energy minima in the original energy surface, by coupling these groups of minima in the original surface to basins in a highly deformed energy surface, which contains a significantly reduced number of minima. The experimental crystal structures of formamide, imidazole, and maleic and succinic anhydrides were predicted as the global minima of the AMBER potential and were found among the lowest-energy minima for the DISCOVER potential. The results of the predictions serve as tests for evaluating the two potentials and may serve as a guide for potential refinements. Another important goal of this study was to clarify the role of the dipole moment contribution in ca...

Multiple description quantization by deterministic annealing

Abstract: The design of vector quantizers for diversity-based communication over two or more channels of possibly differing capacities and failure probabilities, is considered. The crucial dependence of current design techniques on initialization, especially of index assignment, is well recognized. Instead, we propose to pursue a deterministic annealing approach which is independent of initialization, does not assume any prior knowledge of the source density, and avoids many poor local minima of the cost surface. The approach consists of iterative optimization of a random encoder at gradually decreasing levels of randomness as measured by the Shannon entropy. At the limit of zero entropy, a hard multiple description (MD) quantizer is obtained. This process is directly analogous to annealing processes in statistical physics. Via an alternative derivation, we show that it may also be interpreted as approximating the minimum rate sums among points on the convex hull of the MD achievable rate-distortion region of El Gamal and Cover, subject to constraints on the sizes of the reproduction alphabets. To illustrate the potential of our approach, we present simulation results that show substantial performance gains over existing design techniques.

An online parameter estimator for quick convergence and time-varying linear systems

Abstract: A recursive algorithm called 3-OM is presented to estimate parameters and noise variances for discrete-time linear stochastic systems. The unprojected version of 3-OM is globally convergent with probability 1 to minima of the asymptotic negative log-likelihood function. 3-OM approximates the quick convergence attained by the optimal nonlinear filter used as a parameter estimator. The state-space form of 3-OM permits application to time-varying linear systems and to online tuning of a Kalman filter.

Multicanonical jump walk annealing: An efficient method for geometric optimization

Abstract: A new global optimization method, multicanonical jump walk annealing (MJWA), is proposed and applied to the geometric optimization of Lennard-Jones and Morse clusters and the hydrophobic (B), hydrophilic (L), and neutral (N) (BLN) protein model. The method efficiently finds the global minima of these systems. In four comparative studies, MJWA greatly outperforms the conventional simulated annealing in locating the global minima. Theoretical comparison with other global optimization methods is discussed. Through this paper, we demonstrate a criterion for devising stochastic global optimization schemes. Namely, a stochastic global optimization method must favor the global minimum thermodynamically and at the same time be able to cross the high energy barriers.

Exploring the similarities between potential smoothing and simulated annealing

Abstract: Simulated annealing and potential function smoothing are two widely used approaches for global energy optimization of molecular systems. Potential smoothing as implemented in the diffusion equation method has been applied to study partitioning of the potential energy surface (PES) for N-Acetyl-Ala-Ala-N-Methylamide (CDAP) and the clustering of conformations on deformed surfaces. A deformable version of the united-atom OPLS force field is described, and used to locate all local minima and conformational transition states on the CDAP surface. It is shown that the smoothing process clusters conformations in a manner consistent with the inherent structure of the undeformed PES. Smoothing deforms the original surface in three ways: structural shifting of individual minima, merging of adjacent minima, and energy crossings between unrelated minima. A master equation approach and explicit molecular dynamics trajectories are used to uncover similar features in the equilibrium probability distribution of CDAP minima as a function of temperature. Qualitative and quantitative correlations between the simulated annealing and potential smoothing approaches to enhanced conformational sampling are established. c 2000 John Wiley & Sons, Inc. J Comput Chem 21: 531-552, 2000

Evolution strategies for optical tomographic characterization of homogeneous media.

Abstract: The reconstruction problem in diffuse optical tomography can be formulated as an optimization problem, in which an objective function has to be minimized. Current model-based iterative image reconstruction schemes commonly use information about the gradient of the objective function to locate the minimum. These gradient-based search algorithms often find local minima close to an initial guess, or do not converge if the gradient is very small. If the initial guess is too far from the solution, gradient-based schemes prove inefficient for finding the global minimum. In this work we introduce evolution-strategy (ES) algorithms for diffuse optical tomography. These algorithms seek to find the global minimum and are less sensitive to initial guesses and regions with small gradients. We illustrate the fundamental concepts by comparing the performance of gradient-based schemes and ES algorithms in finding optical properties (absorption coefficient µa, scattering coefficient µs, and anisotropy factor g) of a homogenous medium.

A boundary extraction method based on Dual-T-Snakes and dynamic programming

Abstract: The original proposal of active contour models, also called snakes, for image segmentation, suffers from a strong sensitivity to its initial position and can not deal with topological changes. The sensitivity to initialization can be addressed by dynamic programming (DP) techniques which have the advantage of guaranteeing the global minimum and of being more stable numerically than the variational approaches. Their disadvantages are the storage requirements and computational complexity. In this paper we address these limitations of DP by reducing the region of interest (search space) through the use of the Dual-T-Snake approach. The solution of this method consists of two curves enclosing each object boundary which allows the definition of a more efficient search space for a DP technique. The resulting method (Dual-T-Snake plus DP) inherits the capability of changing the topology and avoiding local minima from the Dual-T-Snake and the global optimal properties of the dynamic programming. It can be also extended for 3D.