Showing papers on "Maxima and minima published in 1998"
TL;DR: The current knowledge about numerical instabilities such as checkerboards, mesh-dependence and local minima occurring in applications of the topology optimization method are summarized and the methods with which they can be avoided are listed.
Abstract: In this paper we seek to summarize the current knowledge about numerical instabilities such as checkerboards, mesh-dependence and local minima occurring in applications of the topology optimization method. The checkerboard problem refers to the formation of regions of alternating solid and void elements ordered in a checkerboard-like fashion. The mesh-dependence problem refers to obtaining qualitatively different solutions for different mesh-sizes or discretizations. Local minima refers to the problem of obtaining different solutions to the same discretized problem when choosing different algorithmic parameters. We review the current knowledge on why and when these problems appear, and we list the methods with which they can be avoided and discuss their advantages and disadvantages.
1,796 citations
01 Dec 1998
TL;DR: A split-and-merge expectation-maximization algorithm to overcome the local maxima problem in parameter estimation of finite mixture models and is applied to the training of gaussian mixtures and mixtures of factor analyzers and shows the practical usefulness by applying it to image compression and pattern recognition problems.
Abstract: We present a split-and-merge expectation-maximization (SMEM) algorithm to overcome the local maxima problem in parameter estimation of finite mixture models. In the case of mixture models, local maxima often involve having too many components of a mixture model in one part of the space and too few in another, widely separated part of the space. To escape from such configurations, we repeatedly perform simultaneous split-and-merge operations using a new criterion for efficiently selecting the split-and-merge candidates. We apply the proposed algorithm to the training of gaussian mixtures and mixtures of factor analyzers using synthetic and real data and show the effectiveness of using the split- and-merge operations to improve the likelihood of both the training data and of held-out test data. We also show the practical usefulness of the proposed algorithm by applying it to image compression and pattern recognition problems.
422 citations
TL;DR: Extensions to DEM capable of correcting its empirical behavior are systematically investigated and potential smoothing is characterized by three salient features, which should serve as useful tools for global energy optimization on a variety of difficult problems of practical interest.
Abstract: Global energy optimization of a molecular system is difficult due to the well-known “multiple minimum” problem. The rugged potential energy surface (PES) characteristic of multidimensional systems can be transformed reversibly using potential smoothing to generate a new surface that is easier to search for favorable configurations. The diffusion equation method (DEM) is one example of a potential smoothing algorithm. Potential smoothing as implemented in DEM is intuitively appealing and has certain appropriate statistical mechanical properties, but often fails to identify the global minimum even for relatively small problems. In the present paper, extensions to DEM capable of correcting its empirical behavior are systematically investigated. Two types of local search (LS) procedures are applied during the reversing schedule from the smooth deformed PES to the undeformed surface. Changes needed to generate smoothable versions of standard molecular mechanics force fields such as AMBER/OPLS and MM2 are also described. The resulting methods are applied in an attempt to determine the global energy minimum for a variety of systems in different coordinate representations. The problems studied include argon clusters, cycloheptadecane, capped polyalanine, and the docking of R-helices. Depending on the specific problem, potential smoothing and search (PSS) is performed in Cartesian, torsional, or rigid body space. For example, PSS finds a very low energy structure for cycloheptadecane with much greater efficiency than a search restricted to the undeformed potential surface. It is shown that potential smoothing is characterized by three salient features. As the level of smoothing is increased, unique minima merge into a common basin, crossings can occur in the relative energies of a pair of minima, and the spatial locations of minima are shifted due to the averaging effects of smoothing. Local search procedures improve the ability of smoothing methods to locate global minima because they facilitate the post facto correction of errors due to energy crossings that may have occurred at higher levels of smoothing. PSS methods should serve as useful tools for global energy optimization on a variety of difficult problems of practical interest.
247 citations
TL;DR: In this paper, a simplified method for calculating advection of mixing ratios in a mass conservative and monotonic manner is presented, which employs a special treatment near local maxima and minima to preserve extremes very well.
173 citations
TL;DR: The reduced gradient following RGF (RGF) algorithm as mentioned in this paper uses the gradient and the Hessian matrix or updates of the latter at every curve point to trace abstract curves connecting stationary points of the potential energy surface PES.
Abstract: The old coordinate driving procedure to find transition structures . in chemical systems is revisited. The well-known gradient criterion, =E x s 0, . which defines the stationary points of the potential energy surface PES , is reduced by one equation corresponding to one search direction. In this manner, abstract curves can be defined connecting stationary points of the PES. Starting at a given minimum, one follows a well-selected coordinate to reach the saddle of interest. Usually, but not necessarily, this coordinate will be related to the . reaction progress. The method, called reduced gradient following RGF , locally has an explicit analytical definition. We present a predictor)corrector method for tracing such curves. RGF uses the gradient and the Hessian matrix or updates of the latter at every curve point. For the purpose of testing a whole surface, the six-dimensional PES of formaldehyde, H CO, was explored by RGF 2 . using the restricted Hartree)Fock RHF method and the STO-3G basis set. Forty-nine minima and saddle points of different indices were found. At least seven stationary points representing bonded structures were detected in addition to those located using another search algorithm on the same level of theory. Further examples are the localization of the saddle for the HCN | CNH . isomerization used for steplength tests and for the ring closure of azidoazo- methine to 1 H-tetrazole. The results show that following the reduced gradient may represent a serious alternative to other methods used to locate saddle points in quantum chemistry. Q 1998 John Wiley & Sons, Inc. J Comput Chem 19: 1087)1100, 1998
171 citations
TL;DR: This paper derives new approaches for applying Lagrangian methods in discrete space, shows that an equilibrium is reached when a feasible assignment to the original problem is found and presents heuristic algorithms to look for equilibrium points, and proposes a new discrete Lagrange-multiplier-based global-search method (DLM) for solving satisfiability problems.
Abstract: Satisfiability is a class of NP-complete problems that model a wide range of real-world applications. These problems are difficult to solve because they have many local minima in their search space, often trapping greedy search methods that utilize some form of descent. In this paper, we propose a new discrete Lagrange-multiplier-based global-search method (DLM) for solving satisfiability problems. We derive new approaches for applying Lagrangian methods in discrete space, we show that an equilibrium is reached when a feasible assignment to the original problem is found and present heuristic algorithms to look for equilibrium points. Our method and analysis provides a theoretical foundation and generalization of local search schemes that optimize the objective alone and penalty-based schemes that optimize the constraints alone. In contrast to local search methods that restart from a new starting point when a search reaches a local trap, the Lagrange multipliers in DLM provide a force to lead the search out of a local minimum and move it in the direction provided by the Lagrange multipliers. In contrast to penalty-based schemes that rely only on the weights of violated constraints to escape from local minima, DLM also uses the value of an objective function (in this case the number of violated constraints) to provide further guidance. The dynamic shift in emphasis between the objective and the constraints, depending on their relative values, is the key of Lagrangian methods. One of the major advantages of DLM is that it has very few algorithmic parameters to be tuned by users. Besides the search procedure can be made deterministic and the results reproducible. We demonstrate our method by applying it to solve an extensive set of benchmark problems archived in DIMACS of Rutgers University. DLM often performs better than the best existing methods and can achieve an order-of-magnitude speed-up for some problems.
147 citations
TL;DR: This work examines combining gradient descent with the global optimization technique of simulated annealing (SA), and shows the resulting algorithm, SARPROP, to be able to escape local minima, but is also able to maintain, and often improve the training times of the RPROP algorithm.
Abstract: A problem with gradient descent algorithms is that they can converge to poorly performing local minima. Global optimization algorithms address this problem, but at the cost of greatly increased training times. This work examines combining gradient descent with the global optimization technique of simulated annealing (SA). Simulated annealing in the form of noise and weight decay is added to resiliant backpropagation (RPROP), a powerful gradient descent algorithm for training feedforward neural networks. The resulting algorithm, SARPROP, is shown through various simulations not only to be able to escape local minima, but is also able to maintain, and often improve the training times of the RPROP algorithm. In addition, SARPROP may be used with a restart training phase which allows a more thorough search of the error surface and provides an automatic annealing schedule.
128 citations
TL;DR: A global optimization approach based on a branch and bound framework and convexification techniques for general twice differentiable nonlinear optimization problems is proposed for the parameter estimation of nonlinear algebraic models.
Abstract: The estimation of parameters in nonlinear algebraic models through the error-in-variables method has been widely studied from a computational standpoint. The method involves the minimization of a weighted sum of squared errors subject to the model equations. Due to the nonlinear nature of the models used, the resulting formulation is nonconvex and may contain several local minima in the region of interest. Current methods tailored for this formulation, although computationally efficient, can only attain convergence to a local solution. In this paper, a global optimization approach based on a branch and bound framework and convexification techniques for general twice differentiable nonlinear optimization problems is proposed for the parameter estimation of nonlinear algebraic models. The proposed convexification techniques exploit the mathematical properties of the formulation. Classical nonlinear estimation problems were solved and will be used to illustrate the various theoretical and computational aspec...
102 citations
TL;DR: A stereo correspondence method by minimizing intensity and gradient errors simultaneously by parameterizing the disparity function by hierarchical Gaussians to avoid local minima in the function minimization.
Abstract: We propose a stereo correspondence method by minimizing intensity and gradient errors simultaneously. In contrast to conventional use of image gradients, the gradients are applied in the deformed image space. Although a uniform smoothness constraint is imposed, it is applied only to nonfeature regions. To avoid local minima in the function minimization, we propose to parameterize the disparity function by hierarchical Gaussians. Both the uniqueness and the ordering constraints can be easily imposed in our minimization framework. Besides, we propose a method to estimate the disparity map and the camera response difference parameters simultaneously. Experiments with various real stereo images show robust performances of our algorithm.
97 citations
TL;DR: This work presents an approach to simplification of scalar fields over unstructured grids which preserves the topology of functions defined over the triangulation, in addition to bounding of the errors.
95 citations
11 Oct 1998
TL;DR: A robust fully non-supervised method dedicated to the segmentation of the brain in T1-weighted MR images and an heuristics using this representation to infer statistics on grey and white matter grey level values from the histogram is proposed.
Abstract: In this paper, we propose a robust fully non-supervised method dedicated to the segmentation of the brain in T1-weighted MR images. The first step consists in the analysis of the scale-space of the histogram first and second derivative. We show first that the crossings in scale-space of trajectories of extrema of different derivative orders follow regular topological properties. These properties allow us to design a new structural representation of a 1D signal. Then we propose an heuristics using this representation to infer statistics on grey and white matter grey level values from the histogram. These statistics are used by an improved morphological process combining two opening sizes to segment the brain. The method has been validated with 70 images coming from 3 different scanners and acquired with various MR sequences.
TL;DR: The magnetoresistance of a narrow single quantum well is spectacularly different from the usual behavior as mentioned in this paper, and the formation of the maxima takes place on very large time scales which suggests a close relation with nuclear spins.
Abstract: The magnetoresistance of a narrow single quantum well is spectacularly different from the usual behavior. At filling factors 2 / 3 and 3 / 5 we observe large and sharp maxima in the longitudinal resistance instead of the expected minima. The peak value of the resistance exceeds those of the surrounding magnetic field regions by a factor of up to 3. The formation of the maxima takes place on very large time scales which suggests a close relation with nuclear spins. We discuss the properties of the observed maxima due to a formation of domains of different electronic states.
01 Jun 1998
TL;DR: In this paper, the local minima-free conditions of the outer-supervised feedforward neural networks (FNN) based on batch-style learning are studied by means of the embedded subspace method.
Abstract: In this paper, the local minima-free conditions of the outer-supervised feedforward neural networks (FNN) based on batch-style learning are studied by means of the embedded subspace method. It is proven that only if the rendition that the number of the hidden neurons is not less than that of the training samples, which is sufficient but not necessary, is satisfied, the network will necessarily converge to the global minima with null cost, and that the condition that the range space of the outer-supervised signal matrix is included in the range space of the hidden output matrix Is sufficient and necessary condition for the local minima-free in the error surface. In addition, under the condition of the number of the hidden neurons being less than that of the training samples and greater than the number of the output neurons, it is demonstrated that there will also only exist the global minima with null cost in the error surface if the first layer weights are adequately selected.
TL;DR: In this paper, an annealing-simplex method was proposed to estimate unsaturated hydraulic parameters in a nonlinear optimization problem with multiple parameters, where the objective functions involved are often topographically complex and may contain many local minima.
Abstract: Inverse estimation of unsaturated hydraulic parameters is often a highly nonlinear optimization problem with multiple parameters. The objective functions involved are often topographically complex and may contain many local minima. Because of these reasons, the inverse solutions are commonly very sensitive to the initial guess of the parameters when conventional optimizers are used. This paper presents an annealing-simplex method that incorporates simulated annealing strategies into a classical downhill simplex method. An upward infiltration experiment was used as an example of inverse estimation to test the method. Numerical experiments of both minimizing an algebraic function and inversion of upward infiltration data showed that the new method successfully converged to the global minimum in all cases, irrespective of the initial hydraulic parameter estimates, while the classical downhill method often converged to unfavorable local minima. The CPU times needed for the annealing-simplex method to estimate 5 and 7 hydraulic parameters simultaneously are about a half hour and 1 hour on a PC, respectively. Additionally, no special requirements are imposed on the objective function, and the method is independent of the details of the simulation submodel. Therefore the proposed method should be applicable to other optimization problems in water resources when it is important to have a robust global search capability.
DOI•
01 Jan 1998
TL;DR: This work introduces some recent and very recent smoothing methods which focus on the preservation of boundaries, spikes and canyons in presence of noise and argues that the nonlinear filters can be interpreted as variants of gradient descent on the loss functions.
Abstract: We introduce some recent and very recent smoothing methods which focus on the preservation of boundaries, spikes and canyons in presence of noise. We try to point out basic principles they have in common; the most important one is the robustness aspect. It is reflected by the use of `cup functions' in the statistical loss functions instead of squares; such cup functions were introduced early in robust statistics to down weight outliers. Basically, they are variants of truncated squares. We discuss all the methods in the common framework of `energy functions', i.e we associate to (most of) the algorithms a `loss function' in such a fashion that the output of the algorithm or the `estimate' is a global or local minimum of this loss function. The third aspect we pursue is the correspondence between loss functions and their local minima and nonlinear filters. We shall argue that the nonlinear filters can be interpreted as variants of gradient descent on the loss functions. This way we can show that some (robust) M-estimators and some nonlinear filters produce almost the same result.
TL;DR: In this paper, a swarm of molecules is used to explore conformational space and a cooperative search algorithm is introduced, which uses multiple, different starting conformations and propagates them in time by integration of Newton's equation of motion.
Abstract: A simulation algorithm is introduced, which uses a swarm of molecules to explore conformational space. The method uses multiple, different starting conformations and propagates them in time by integration of Newton's equation of motion. In contrast to conventional molecular dynamics simulation of a set of independent molecules, in this method each molecule of the swarm is in addition subject to an artificial field that keeps the trajectory of individual molecules tied to the average trajectory of the swarm. In this manner, a search for the global energy minima of many molecules is transformed into a cooperative search. It is shown that such a cooperative search is less attracted by local minima in the potential energy surface and that the total system is more likely to follow an overall potential energy gradient toward the global energy minima.
TL;DR: A new methodology for analysis is presented, based upon consideration of trajectories through weight space by which a training algorithm might escape a hypothesized local minimum, and this method is applied to the well known XOR problem, which has previously been considered to exhibit local minima.
TL;DR: A numerical potential function for point-robot path planning in configuration space based on the theory of fluid mechanics is presented, which creates an environment which is not only free of local minima but also beneficial for navigation control.
Abstract: This paper presents a numerical potential function for point-robot path planning in configuration space based on the theory of fluid mechanics. Ideal fluid is first simulated using Poisson’s equation and heuristic path planning algorithms are established by comparisons of the velocity potentials. Several computational techniques are experimented and compared. A bitmap collision detection technique is proposed for non-point robots. This fluid model creates an environment which is not only free of local minima but also beneficial for navigation control.
TL;DR: This study addresses the latter problem and proposes a modified back-propagation method to keep the sigmoid derivative relatively large while some of the error signals are large and results of numerical experiments substantiate the validity of the method.
TL;DR: In this article, the hybrid numerical optimization technique which incorporates the GA into the Modified Powell method was proposed, which can escape from trapping in the local minimum, and seek out the global minimum with considerably fast convergence.
TL;DR: In this paper, two methods of global minimization, the diffusion equation method and the distance scaling method, are applied to predict the crystal structures of the hexasulfur and benzene molecules.
Abstract: Two methods of global minimization, the diffusion equation method and the distance scaling method, are applied to predict the crystal structures of the hexasulfur and benzene molecules. No knowledge about the systems other than the geometry of the molecules and the pairwise potentials is assumed; i.e., no assumptions are made about the space groups, cell dimensions, or number of molecules in the unit cell. Both methods are based on smoothing transformations of the original potential energy surface, which remove all insignificant local minima; the surviving minima are traced back to the original potential energy surface during the so-called reversing procedure, in which the transformations are gradually removed. The crystal structures, known from experiment, were predicted correctly. To verify the power of the methods, the problem of global minimization of the potential energy of crystals of both molecules was intentionally increased considerably in complexity: viz., the numbers of molecules in the unit c...
12 May 1998
TL;DR: A general affine scaling optimization algorithm is given that converges to a sparse solution for measures chosen from within this subclass of the Schur-concave functions.
Abstract: A general framework based on majorization, Schur-concavity, and concavity is given that facilitates the analysis of algorithm performance and clarifies the relationships between existing proposed diversity measures useful for best basis selection. Admissible sparsity measures are given by the Schur-concave functions, which are the class of functions consistent with the partial ordering on vectors known as majorization. Concave functions form an important subclass of the Schur-concave functions which attain their minima at sparse solutions to the basis selection problem. Based on a particular functional factorization of the gradient, we give a general affine scaling optimization algorithm that converges to a sparse solution for measures chosen from within this subclass.
TL;DR: The standard operators are applied to connected sets that form maxima and minima, which are new, powerful, general tools for analysing and representing images.
Abstract: Mathematical morphology is the analysis of signals and images in terms of shape. Much is based on simple positive Boolean functions that are used to produce filters for binary and greyscale signals and images. In a previous development, the standard operators are applied to connected sets that form maxima and minima. These are new, powerful, general tools for analysing and representing images.
TL;DR: In this article, a study of the multiple minima problem using simulations was performed for a simple case using simulations, and three ways to overcome this problem are suggested: using another estimator, reducing the sensitivity of the data or performing a grid search.
TL;DR: A modification of that algorithm that makes it converge for many multiband filter designs is described, although no proof of convergence is given, the modified algorithm remains simple and converges rapidly in many cases.
Abstract: In a previous paper, we described a constrained least square approach to FIR filter design that does not use "don't care" regions. In that paper, we described a simple algorithm for the design of lowpass filters according to that approach. In this paper, we describe a modification of that algorithm that makes it converge for many multiband filter designs. Although no proof of convergence is given, the modified algorithm remains simple and converges rapidly in many cases. In this approach, the user supplies a lower and upper bound constraint that is exactly satisfied by the local minima and maxima of the frequency response amplitude. Yet, the constraints can be made as tight as desired-the transition band automatically adjusts (widens) to accommodate the constraints.
TL;DR: In this article, the energy landscape of two dimensional network models for covalent glasses is investigated by means of the lid algorithm, which exhaustively analyzes many configuration space regions enclosing deep-lying energy minima.
Abstract: We investigate the energy landscape of two dimensional network models for covalent glasses by means of the lid algorithm. For three different particle densities and for a range of network sizes, we exhaustively analyse many configuration space regions enclosing deep-lying energy minima. We extract the local densities of states and of minima, and the number of states and minima accessible below a certain energy barrier, the 'lid'. These quantities show on average a close to exponential growth as a function of their respective arguments. We calculate the configurational entropy for these pockets of states and find that the excess specific heat exhibits a peak at a critical temperature associated with the exponential growth in the local density of states, a feature of the specific heat also observed in real glasses at the glass transition.
TL;DR: It is proved that all stationary points with finite weights are saddle points with positive error or absolute minima with error zero, so, for finite weights no local minima occur.
TL;DR: In this paper, two generalized trajectory methods are combined to provide a novel and powerful numerical procedure for systematically finding multiple local extrema of a multivariable objective function, which can form part of a strategy for global optimization in which the greatest local maximum and least local minimum in the interior of a specified region are compared to the largest and smallest values of the objective function on the boundary of the region.
Abstract: Two generalized trajectory methods are combined to provide a novel and powerful numerical procedure for systematically finding multiple local extrema of a multivariable objective function. This procedure can form part of a strategy for global optimization in which the greatest local maximum and least local minimum in the interior of a specified region are compared to the largest and smallest values of the objective function on the boundary of the region. The first trajectory method, a homotopy scheme, provides a globally convergent algorithm to find a stationary point of the objective function. The second trajectory method, a relaxation scheme, starts at one stationary point and systematically connects other stationary points in the specified region by a network of trjectories. It is noted that both generalized trajectory methods actually solve the stationarity conditions, and so they can also be used to find multiple roots of a set of nonlinear equations.
01 Apr 1998
TL;DR: Since the Patrick-Fisher (PF) distance is a highly nonlinear function, this work proposes a recursive optimization procedure for searching the directions corresponding to several large local maxima of the PF distance.
Abstract: A method for the linear discrimination of two classes is presented. It searches for the discriminant direction which maximizes the Patrick-Fisher (PF) distance between the projected class-conditional densities. It is a nonparametric method, in the sense that the densities are estimated from the data. Since the PF distance is a highly nonlinear function, we propose a recursive optimization procedure for searching the directions corresponding to several large local maxima of the PF distance. Its novelty lies in the transformation of the data along a found direction into data with deflated maxima of the PF distance and iteration to obtain the next direction. A simulation study and a medical data analysis indicate the potential of the method to find the sequence of directions with significant class separations.
TL;DR: In this paper, the energy landscape of two-dimensional network models for covalent glasses is investigated by means of the lid algorithm, and the authors exhaustively analyse many configuration space regions enclosing deep-lying energy minima.
Abstract: We investigate the energy landscape of two-dimensional network models for covalent glasses by means of the lid algorithm. For three different particle densities and for a range of network sizes, we exhaustively analyse many configuration space regions enclosing deep-lying energy minima. We extract the local densities of states and of minima, and the number of states and minima accessible below a certain energy barrier, the `lid'. These quantities show on average a close to exponential growth as a function of their respective arguments. We calculate the configurational entropy for these pockets of states and find that the excess specific heat exhibits a peak at a critical temperature associated with the exponential growth in the local density of states, a feature of the specific heat also observed in real glasses at the glass transition.