Showing papers on "Maxima and minima published in 2002"

Escaping free-energy minima

Abstract: We introduce a powerful method for exploring the properties of the multidimensional free energy surfaces (FESs) of complex many-body systems by means of coarse-grained non-Markovian dynamics in the space defined by a few collective coordinates. A characteristic feature of these dynamics is the presence of a history-dependent potential term that, in time, fills the minima in the FES, allowing the efficient exploration and accurate determination of the FES as a function of the collective coordinates. We demonstrate the usefulness of this approach in the case of the dissociation of a NaCl molecule in water and in the study of the conformational changes of a dialanine in solution.

An analysis of particle swarm optimizers

Abstract: Many scientific, engineering and economic problems involve the optimisation of a set of parameters. These problems include examples like minimising the losses in a power grid by finding the optimal configuration of the components, or training a neural network to recognise images of people's faces. Numerous optimisation algorithms have been proposed to solve these problems, with varying degrees of success. The Particle Swarm Optimiser (PSO) is a relatively new technique that has been empirically shown to perform well on many of these optimisation problems. This thesis presents a theoretical model that can be used to describe the long-term behaviour of the algorithm. An enhanced version of the Particle Swarm Optimiser is constructed and shown to have guaranteed convergence on local minima. This algorithm is extended further, resulting in an algorithm with guaranteed convergence on global minima. A model for constructing cooperative PSO algorithms is developed, resulting in the introduction of two new PSO-based algorithms. Empirical results are presented to support the theoretical properties predicted by the various models, using synthetic benchmark functions to investigate specific properties. The various PSO-based algorithms are then applied to the task of training neural networks, corroborating the results obtained on the synthetic benchmark functions.

HAMMER: hierarchical attribute matching mechanism for elastic registration

Abstract: A new approach is presented for elastic registration of medical images, and is applied to magnetic resonance images of the brain. Experimental results demonstrate very high accuracy in superposition of images from different subjects. There are two major novelties in the proposed algorithm. First, it uses an attribute vector, i.e., a set of geometric moment invariants (GMIs) that are defined on each voxel in an image and are calculated from the tissue maps, to reflect the underlying anatomy at different scales. The attribute vector, if rich enough, can distinguish between different parts of an image, which helps establish anatomical correspondences in the deformation procedure; it also helps reduce local minima, by reducing ambiguity in potential matches. This is a fundamental deviation of our method, referred to as the hierarchical attribute matching mechanism for elastic registration (HAMMER), from other volumetric deformation methods, which are typically based on maximizing image similarity. Second, in order to avoid being trapped by local minima, i.e., suboptimal poor matches, HAMMER uses a successive approximation of the energy function being optimized by lower dimensional smooth energy functions, which are constructed to have significantly fewer local minima. This is achieved by hierarchically selecting the driving features that have distinct attribute vectors, thus, drastically reducing ambiguity in finding correspondence. A number of experiments demonstrate that the proposed algorithm results in accurate superposition of image data from individuals with significant anatomical differences.

Radius margin bounds for support vector machines with the RBF kernel

Abstract: An important approach for efficient support vector machine (SVM) model selection is to use differentiable bounds of the leave-one-out (LOO) error. Past efforts focused on finding tight bounds of LOO. However, their practical viability is still not very satisfactory. Duan et al. (2002) has been shown that radius margin bound gives good prediction for L2-SVM. In this paper, through the analyses why this bound performs well for L2-SVM, we show that finding a bound whose minima are in a region with small LOO values may be more important than its tightness. Based on this principle we propose modified radius margin bounds for L1-SVM where the original bound is only applicable to the hard-margin case. Our modification for L1-SVM achieves comparable performance to L2-SVM.

Stable Fixed Points of Loopy Belief Propagation Are Local Minima of the Bethe Free Energy

Abstract: We extend recent work on the connection between loopy belief propagation and the Bethe free energy. Constrained minimization of the Bethe free energy can be turned into an unconstrained saddle-point problem. Both converging double-loop algorithms and standard loopy belief propagation can be interpreted as attempts to solve this saddle-point problem. Stability analysis then leads us to conclude that stable fixed points of loopy belief propagation must be (local) minima of the Bethe free energy. Perhaps surprisingly, the converse need not be the case: minima can be unstable fixed points. We illustrate this with an example and discuss implications.

A general backpropagation algorithm for feedforward neural networks learning

Abstract: A general backpropagation algorithm is proposed for feedforward neural network learning with time varying inputs. The Lyapunov function approach is used to rigorously analyze the convergence of weights, with the use of the algorithm, toward minima of the error function. Sufficient conditions to guarantee the convergence of weights for time varying inputs are derived. It is shown that most commonly used backpropagation learning algorithms are special cases of the developed general algorithm.

Taboo Search: An Approach to the Multiple-Minima Problem for Continuous Functions

Abstract: We decribe an approach, based on Taboo (or “Tabu”) Search for discrete functions, for solving the multiple-minima problem of continuous functions. As demonstrated by model calculations, the algorithm avoids entrapment in local minima and continues the search to give a near-optimal final solution. The procedure is generally applicable, derivative-free, easy to implement, conceptually simpler than Simulated Annealing and open to further improvement.

Geometric approach to the dynamic glass transition.

Abstract: We numerically study the potential energy landscape of a fragile glassy system and find that the dynamic crossover corresponding to the glass transition is actually the effect of an underlying geometric transition caused by the vanishing of the instability index of saddle points of the potential energy. Furthermore, we show that the potential energy barriers connecting local glassy minima increase with decreasing energy of the minima, and we relate this behavior to the fragility of the system. Finally, we analyze the real space structure of activated processes by studying the distribution of particle displacements for local minima connected by simple saddles.

The generalized equivalent uniform dose function as a basis for intensity-modulated treatment planning.

Abstract: The efficiency of intensity-modulated radiation therapy (IMRT) treatment planning depends critically on the presence or absence of multiple local minima in the feasible search space. We analyse the convexity of the generalized equivalent uniform dose equation (Niemierko A 1999 Med. Phys. 26 1100) when used either in the objective function or in the constraints. The practical importance of this analysis is that convex objective functions minimized over convex feasibility spaces do not have multiple local minima, likewise for concave objective functions maximized over convex feasibility spaces. Both of these situations are referred to as 'convex problems' and computationally efficient local search methods can be used for their solution. We also show that the Poisson-based tumour control probability objective function is strictly concave (if one neglects inter-patient heterogeneity), and hence it implies a single local minimum if maximized over a convex feasibility space. Even when including inter-patient heterogeneity, multiple local minima, although theoretically possible, are expected to be of minimal concern. The generalized equivalent uniform dose function (EUDa) is proved to be convex or concave depending on its only parameter a: when a is equal to or greater than 1, minimizing EUDa, on a convex feasibility space leads to a single minimum; when a is less than 1, maximizing EUDa, on a convex feasibility space leads to a single minimum. We also study a recently proposed practical, yet difficult, IMRT treatment planning formulation: unconstrained optimization of the objective function proposed by Wu et al (2002 Int. J. Radiat. Oncol. Biol. Phys. 52 224-35), which is expressed in terms of the EUDa for the target and normal tissues. This formulation may theoretically lead to multiple local minima. We propose a procedure for improving resulting solutions based on the convexity properties of the underlying objective function terms.

Template matching based object recognition with unknown geometric parameters

Abstract: We examine the problem of locating an object in an image when size and rotation are unknown. Previous work has shown that with known geometric parameters, an image restoration method can be useful by estimating a delta function at the object location. When the geometric parameters are unknown, this method becomes impractical because the likelihood surface to be minimized across size and rotation has numerous local minima and areas of zero gradient. We propose a new approach where a smooth approximation of the template is used to minimize a well-behaved likelihood surface. A coarse-to-fine approximation of the original template using a diffusion-like equation is used to create a library of templates. Using this library, we can successively perform minimizations which are locally well-behaved. As detail is added to the template, the likelihood surface gains local minima, but previous estimates place us within a well-behaved "bowl" around the global minimum, leading to an accurate estimate. Numerical experiments are shown which verify the value of this approach for a wide range of values of the geometric parameters.

Weak sharp minima revisited Part I: basic theory

Abstract: The notion of sharp minima, or strongly unique lo­ cal miuillla, elll erged in t he late 1970's as an important tool in the a ualysis of the perturbation behavior of certa in cl asses of optimi~a ­

Data perturbation for escaping local maxima in learning

Abstract: Almost all machine learning algorithms--be they for regression, classification or density estimation--seek hypotheses that optimize a score on training data. In most interesting cases, however, full global optimization is not feasible and local search techniques are used to discover reasonable solutions. Unfortunately, the quality of the local maxima reached depends on initialization and is often weaker than the global maximum. In this paper, we present a simple approach for combining global search with local optimization to discover improved hypotheses in general machine learning problems. The main idea is to escape local maxima by perturbing the training data to create plausible new ascent directions, rather than perturbing hypotheses directly. Specifically, we consider example-reweighting strategies that are reminiscent of boosting and other ensemble learning methods, but applied in a different way with a different goal: to produce a single hypothesis that achieves a good score on training and test data. To evaluate the performance of our algorithms we consider a number of problems in learning Bayesian networks from data, including discrete training problems (structure search), continuous training problems (parametric EM, non-linear logistic regression), and mixed training problems (Structural EM)- on both synthetic and real-world data. In each case, we obtain state of the art performance on both training and test data.

Multiple local minima in IMRT optimization based on dose-volume criteria.

Abstract: Multiple local minima traps are known to exist in dose-volume and dose-response objective functions. Nevertheless, their presence and consequences are not considered impediments in finding satisfactory solutions in routine optimization of IMRT plans using gradient methods. However, there is often a concern that a significantly superior solution may exist unbeknownst to the planner and that the optimization process may not be able to reach it. We have investigated the soundness of the assumption that the presence of multiple minima traps can be ignored. To find local minima, we start the optimization process a large number of times with random initial intensities. We investigated whether the occurrence of local minima depends upon the choice of the objective function parameters and the number of variables and whether their existence is an impediment in finding a satisfactory solution. To learn about the behavior of multiple minima, we first used a symmetric cubic phantom containing a cubic target and an organ-at-risk surrounding it to optimize the beam weights of two pairs of parallel-opposed beams using a gradient technique. The phantom studies also served to test our software. Objective function parameters were chosen to ensure that multiple minima would exist. Data for 500 plans, optimized with random initial beam weights, were analyzed. The search process did succeed in finding the local minima and showed that the number of minima depends on the parameters of the objective functions. It was also found that the consequences of local minima depended on the number of beams. We further searched for the multiple minima in intensity-modulated treatment plans for a head-and-neck case and a lung case. In addition to the treatment plan scores and the dose-volume histograms, we examined the dose distributions and intensity patterns. We did not find any evidence that multiple local minima affect the outcome of optimization using gradient techniques in any clinically significant way. Our study supports the notion that multiple minima should not be an impediment to finding a good solution when gradient-based optimization techniques are employed. Changing the parameters for the objective function had no observable effect on our findings.

Method for the estimation and recovering from general affine transforms in digital watermarking applications

Abstract: An important problem constraining the practical exploitation of robust watermarking technologies is the low robustness of the existing algorithms against geometrical distortions such as rotation, scaling, cropping, translation, change of aspect ratio and shearing. All these attacks can be uniquely described by general affine transforms. In this work, we propose a robust estimation method using apriori known regularity of a set of points. These points can be typically local maxima, or peaks, resulting either from the autocorrelation function (ACF) or from the magnitude spectrum (MS) generated by periodic patterns, which result in regularly aligned and equally spaced points. This structure is kept under any affine transform. The estimation of affine transform parameters is formulated as a robust penalized Maximum Likelihood (ML) problem. We propose an efficient approximation of this problem based on Hough transform (HT) or Radon transform (RT), which are known to be very robust in detecting alignments, even when noise is introduced by misalignments of points, missing points, or extra points. The high efficiency of the method is demonstrated even when severe degradations have occurred, including JPEG compression with a quality factor of 50%, where other known algorithms fail. Results with the Stirmark benchmark confirm the high robustness of the proposed method.

New Classes of Globally Convexized Filled Functions for Global Optimization

Abstract: We propose new classes of globally convexized filled functions Unlike the globally convexized filled functions previously proposed in literature, the ones proposed in this paper are continuously differentiable and, under suitable assumptions, their unconstrained minimization allows to escape from any local minima of the original objective function Moreover we show that the properties of the proposed functions can be extended to the case of box constrained minimization problems We also report the results of a preliminary numerical experience

Early computation of part structure: Evidence from visual search

Abstract: The visual system represents object shapes in terms of intermediate-level parts. The minima rule proposes that the visual system uses negative minima of curvature to define boundaries between parts. We used visual search to test whether part structures consistent with the minima rule are computed preattentively—or at least, rapidly and early in visual processing. The results of Experiments 1 and 2 showed that whereas the search for a non-minima-segmented shape is fast and efficient among minimasegmented shapes, the reverse search is slow and inefficient. This asymmetry is expected if parsing at negative minima occurs obligatorily. The results of Experiments 3 and 4 showed that although both minima- and non-minima-segmented shapes pop out among unsegmented shapes, the search for minimasegmented shapes is significantly slower. Together, these results demonstrate that the visual system segments shapes into parts, using negative minima of curvature, and that it does so rapidly in early stages of visual processing.

Grain Filters

Abstract: Motivated by operators simplifying the topographic map of a function, we study the theoretical properties of two kinds of “grain” filters. The first category, discovered by L. Vincent, defines grains as connected components of level sets and removes those of small area. This category is composed of two filters, the maxima filter and the minima filter. However, they do not commute. The second kind of filter, introduced by Masnou, works on “shapes”, which are based on connected components of level sets. This filter has the additional property that it acts in the same manner on upper and lower level sets, that is, it commutes with an inversion of contrast. We discuss the relations of Masnou's filter with other classes of connected operators introduced in the literature. We display some experiments to show the main properties of the filters discussed above and compare them.

Use of covariance analysis for the prediction of structural domain boundaries from multiple protein sequence alignments.

Abstract: Current methods for identification of domains within protein sequences require either structural information or the identification of homologous domain sequences in different sequence contexts. Knowledge of structural domain boundaries is important for fold recognition experiments and structural determination by X-ray crystallography or nuclear magnetic resonance spectroscopy using the divideand-conquer approach. Here, a new and conceptually simple method for the identification of structural domain boundaries in multiple protein sequence alignments is presented. Analysis of covariance at positions within the alignment is first used to predict 3D contacts. By the nature of the domain as an independent folding unit, inter-domain predicted contacts are fewer than intra-domain predicted contacts. By analysing all possible domain boundaries and constructing a smoothed profile of predicted contact density (PCD), true structural domain boundaries are predicted as local profile minima associated with low PCD. A training data set is constructed from 52 non-homologous twodomain protein sequences of known 3D structure and used to determine optimal parameters for the profile analysis. The alignments in the training data set contained 48 17 (mean SD) sequences and lengths of 257 121 residues. Of the 47 alignments yielding predictions, 35% of true domain boundaries are predicted to within 15 amino acids by the local profile minimum with the lowest profile value. Including predictions from the second- and third-lowest local minima increases the correct domain boundary coverage to 60%, whereas the lowest five local minima cover 79% of correct domain boundaries. Through further profile analysis, criteria are presented which reliably identify subsets of more accurate predictions. Retrospective analysis of CASP3 targets shows predictions of sufficient accuracy to enable dramatically improved fold recognition results. Finally, a prediction is made for geminivirus AL1 protein which is in full agreement with biochemical data, yielding a plausible, novel threading result.

An adjoint method for the assimilation of statistical characteristics into eddy-resolving ocean models

Abstract: The study investigates perspectives of the parameter estimation problem with the adjoint method in eddy-resolving models. Sensitivity to initial conditions resulting from the chaotic nature of this type of model limits the direct application of the adjoint method by predictability. Prolonging the period of assimilation is accompanied by the appearance of an increasing number of secondary minima of the cost function that prevents the convergence of this method. In the framework of the Lorenz model it is shown that averaged quantities are suitable for describing invariant properties, and that secondary minima are for this type of data transformed into stochastic deviations. An adjoint method suitable for the assimilation of statistical characteristics of data and applicable on time scales beyond the predictability limit is presented. The approach assumes a greater predictability for averaged quantities. The adjoint to a prognostic model for statistical moments is employed for calculating cost function gradients that ignore the fine structure resulting from secondary minima. Coarse resolution versions of eddy-resolving models are used for this purpose. Identical twin experiments are performed with a quasigeostrophic model to evaluate the performance and limitations of this approach in improving models by estimating parameters. The wind stress curl is estimated from a simulated mean stream function. A very simple parameterization scheme for the assimilation of second-order moments is shown to permit the estimation of gradients that perform efficiently in minimizing cost functions.

On the Stationary Points of the Squared Distance between Two Ellipses with a Common Focus

Abstract: In this paper we introduce an effective algebraic method for the computation of all the stationary points of the squared distance $d^2$ between a point on one ellipse and a point on a second ellipse with a focus in common with the first one. This problem comes from celestial mechanics, in which the minima between two elliptic Keplerian orbits are relevant to study the probability of collision; some applications of our algorithm in this field are shown. This algorithm is based on the use of the fast Fourier transform to obtain the coefficients of the resultant of the two bivariate components of the gradient of $d^2$ with respect to one variable and relies on specific tools in symbolic computation. An upper bound to the total number of stationary points that we have to expect in this problem is also given; this is done using some tools from algebraic geometry.

Studies of multi‐start clustering for global optimization

Abstract: Global/multi-modal optimization problems arise in many engineering applications. Owing to the existence of multiple minima, it is a challenge to solve the multi-modal optimization problem and to identify the global minimum especially if efficiency is a concern. In this paper, variants of the multi-start with clustering strategy are developed and studied for identifying multiple local minima in nonlinear global optimization problems. The study considers the sampling procedure, the use of Hessian information in forming clusters, the technique for cluster analysis and the local search procedure. Variations of multi-start with clustering are applied to 15 multi-modal problems. A comparative study focuses on the overall search effectiveness in terms of the number of local searches performed, local minima found and required function evaluations. The performance of these multi-start clustering algorithms ranges from very efficient to very robust. Copyright © 2002 John Wiley & Sons, Ltd.