Surrogate-based infill optimization applied to electromagnetic problems
01 Sep 2010-International Journal of Rf and Microwave Computer-aided Engineering (Wiley Subscription Services, Inc., A Wiley Company)-Vol. 20, Iss: 5, pp 492-501
TL;DR: The expected improvement approach is demonstrated on two electromagnetic problems, namely, a microwave filter and a textile antenna, and it is shown that this approach can improve the quality of designs on these problems.
Abstract: The increasing use of expensive computer simulations in engineering places a serious computational burden on associated optimization problems. Surrogate-based optimization becomes standard practice in analyzing such expensive black-box problems. This article discusses several approaches that use surrogate models for optimization and highlights one sequential design approach in particular, namely, expected improvement. The expected improvement approach is demonstrated on two electromagnetic problems, namely, a microwave filter and a textile antenna. © 2010 Wiley Periodicals, Inc. Int J RF and Microwave CAE, 2010.
Citations
More filters
6,278 citations
TL;DR: The authors propose the efficient multiobjective optimization (EMO) algorithm which uses Kriging models and multi objective versions of the probability of improvement and expected improvement criteria to identify the Pareto front with a minimal number of expensive simulations.
Abstract: The use of surrogate based optimization (SBO) is widely spread in engineering design to reduce the number of computational expensive simulations. However, "real-world" problems often consist of multiple, conflicting objectives leading to a set of competitive solutions (the Pareto front). The objectives are often aggregated into a single cost function to reduce the computational cost, though a better approach is to use multiobjective optimization methods to directly identify a set of Pareto-optimal solutions, which can be used by the designer to make more efficient design decisions (instead of weighting and aggregating the costs upfront). Most of the work in multiobjective optimization is focused on multiobjective evolutionary algorithms (MOEAs). While MOEAs are well-suited to handle large, intractable design spaces, they typically require thousands of expensive simulations, which is prohibitively expensive for the problems under study. Therefore, the use of surrogate models in multiobjective optimization, denoted as multiobjective surrogate-based optimization, may prove to be even more worthwhile than SBO methods to expedite the optimization of computational expensive systems. In this paper, the authors propose the efficient multiobjective optimization (EMO) algorithm which uses Kriging models and multiobjective versions of the probability of improvement and expected improvement criteria to identify the Pareto front with a minimal number of expensive simulations. The EMO algorithm is applied on multiple standard benchmark problems and compared against the well-known NSGA-II, SPEA2 and SMS-EMOA multiobjective optimization methods.
197 citations
Cites background from "Surrogate-based infill optimization..."
...Of particular interest are the PoI and EI statistical criteria which are widely used for single-objective optimization [10,24]....
[...]
Journal Article•
TL;DR: An efficient object-oriented Kriging implementation and several Kriged extensions are presented, providing a flexible and easily extendable framework to test and implement new K Riging flavors while reusing as much code as possible.
Abstract: When analyzing data from computationally expensive simulation codes, surrogate modeling methods are firmly established as facilitators for design space exploration, sensitivity analysis, visualization and optimization. Kriging is a popular surrogate modeling technique used for the Design and Analysis of Computer Experiments (DACE). Hence, the past decade Kriging has been the subject of extensive research and many extensions have been proposed, e.g., co-Kriging, stochastic Kriging, blind Kriging, etc. However, few Kriging implementations are publicly available and tailored towards scientists and engineers. Furthermore, no Kriging toolbox exists that unifies several Kriging flavors. This paper addresses this need by presenting an efficient object-oriented Kriging implementation and several Kriging extensions, providing a flexible and easily extendable framework to test and implement new Kriging flavors while reusing as much code as possible.
166 citations
Cites background or methods from "Surrogate-based infill optimization..."
..., optimization of a textile antenna (Couckuyt et al., 2010), identification of the elasticity of the middle-ear drum (Aernouts et al....
[...]
...The ooDACE Toolbox has already been applied successfully to a wide range of problems, e.g., optimization of a textile antenna (Couckuyt et al., 2010), identification of the elasticity of the middle-ear drum (Aernouts et al., 2010), etc....
[...]
TL;DR: A surrogate management strategy based on ensemble learning techniques developed in machine learning is adopted, which builds a large number of surrogate models before optimization and adaptively selects a small yet diverse subset of them during the optimization to achieve the best local approximation accuracy and reduce the computational complexity.
Abstract: In solving many real-world optimization problems, neither mathematical functions nor numerical simulations are available for evaluating the quality of candidate solutions. Instead, surrogate models must be built based on historical data to approximate the objective functions and no new data will be available during the optimization process. Such problems are known as offline data-driven optimization problems. Since the surrogate models solely depend on the given historical data, the optimization algorithm is able to search only in a very limited decision space during offline data-driven optimization. This paper proposes a new offline data-driven evolutionary algorithm to make the full use of the offline data to guide the search. To this end, a surrogate management strategy based on ensemble learning techniques developed in machine learning is adopted, which builds a large number of surrogate models before optimization and adaptively selects a small yet diverse subset of them during the optimization to achieve the best local approximation accuracy and reduce the computational complexity. Our experimental results on the benchmark problems and a transonic airfoil design example show that the proposed algorithm is able to handle offline data-driven optimization problems with up to 100 decision variables.
144 citations
Cites background from "Surrogate-based infill optimization..."
...Whereas online data-driven EAs can use various infill sampling criteria [30], [31] to include additional training data for updating the surrogates during the optimization, offline data-driven EAs have no access to the real fitness evaluations and no model update can be carried out....
[...]
...A class of more formal model management strategies are known as infill sampling criteria [30], [31], which help select the next solution to be evaluated using the expensive fitness...
[...]
TL;DR: This paper aims to present an overview of the state-of-the-art and recently developed LBE-based strategies as applied to the solution of engineering problems in the field of Electromagnetics.
Abstract: There is a wide number of problems in electromagnetic (EM) engineering that require a real-time response or in which the input–output relationship is not a-priori known or cannot be defined due to ...
136 citations
References
More filters
Book•
23 Nov 2005TL;DR: The treatment is comprehensive and self-contained, targeted at researchers and students in machine learning and applied statistics, and deals with the supervised learning problem for both regression and classification.
Abstract: A comprehensive and self-contained introduction to Gaussian processes, which provide a principled, practical, probabilistic approach to learning in kernel machines. Gaussian processes (GPs) provide a principled, practical, probabilistic approach to learning in kernel machines. GPs have received increased attention in the machine-learning community over the past decade, and this book provides a long-needed systematic and unified treatment of theoretical and practical aspects of GPs in machine learning. The treatment is comprehensive and self-contained, targeted at researchers and students in machine learning and applied statistics. The book deals with the supervised-learning problem for both regression and classification, and includes detailed algorithms. A wide variety of covariance (kernel) functions are presented and their properties discussed. Model selection is discussed both from a Bayesian and a classical perspective. Many connections to other well-known techniques from machine learning and statistics are discussed, including support-vector machines, neural networks, splines, regularization networks, relevance vector machines and others. Theoretical issues including learning curves and the PAC-Bayesian framework are treated, and several approximation methods for learning with large datasets are discussed. The book contains illustrative examples and exercises, and code and datasets are available on the Web. Appendixes provide mathematical background and a discussion of Gaussian Markov processes.
11,357 citations
Additional excerpts
...[33]....
[...]
Book•
01 Jan 1991TL;DR: In this paper, the authors present a survey of statistics for spatial data in the field of geostatistics, including spatial point patterns and point patterns modeling objects, using Lattice Data and spatial models on lattices.
Abstract: Statistics for Spatial Data GEOSTATISTICAL DATA Geostatistics Spatial Prediction and Kriging Applications of Geostatistics Special Topics in Statistics for Spatial Data LATTICE DATA Spatial Models on Lattices Inference for Lattice Models SPATIAL PATTERNS Spatial Point Patterns Modeling Objects References Author Index Subject Index.
8,631 citations
TL;DR: This paper introduces the reader to a response surface methodology that is especially good at modeling the nonlinear, multimodal functions that often occur in engineering and shows how these approximating functions can be used to construct an efficient global optimization algorithm with a credible stopping rule.
Abstract: In many engineering optimization problems, the number of function evaluations is severely limited by time or cost. These problems pose a special challenge to the field of global optimization, since existing methods often require more function evaluations than can be comfortably afforded. One way to address this challenge is to fit response surfaces to data collected by evaluating the objective and constraint functions at a few points. These surfaces can then be used for visualization, tradeoff analysis, and optimization. In this paper, we introduce the reader to a response surface methodology that is especially good at modeling the nonlinear, multimodal functions that often occur in engineering. We then show how these approximating functions can be used to construct an efficient global optimization algorithm with a credible stopping rule. The key to using response surfaces for global optimization lies in balancing the need to exploit the approximating surface (by sampling where it is minimized) with the need to improve the approximation (by sampling where prediction error may be high). Striking this balance requires solving certain auxiliary problems which have previously been considered intractable, but we show how these computational obstacles can be overcome.
6,914 citations
"Surrogate-based infill optimization..." refers methods in this paper
...[4, 27] as the Efficient Global Optimization (EGO) algorithm....
[...]
...This paper describes a popular optimization method for expensive black-box simulators based on kriging surrogate models, namely, expected improvement (EI) [4]....
[...]
...Therefore, the original EGO algorithm used kriging [4] as surrogate model of choice, since kriging provides analytical formulae for prediction as well as a point-wise error estimation....
[...]
TL;DR: This paper presents a meta-modelling framework for estimating Output from Computer Experiments-Predicting Output from Training Data and Criteria Based Designs for computer Experiments.
Abstract: Many scientific phenomena are now investigated by complex computer models or codes A computer experiment is a number of runs of the code with various inputs A feature of many computer experiments is that the output is deterministic--rerunning the code with the same inputs gives identical observations Often, the codes are computationally expensive to run, and a common objective of an experiment is to fit a cheaper predictor of the output to the data Our approach is to model the deterministic output as the realization of a stochastic process, thereby providing a statistical basis for designing experiments (choosing the inputs) for efficient prediction With this model, estimates of uncertainty of predictions are also available Recent work in this area is reviewed, a number of applications are discussed, and we demonstrate our methodology with an example
6,583 citations
TL;DR: Cressie et al. as discussed by the authors presented the Statistics for Spatial Data (SDS) for the first time in 1991, and used it for the purpose of statistical analysis of spatial data.
Abstract: 5. Statistics for Spatial Data. By N. Cressie. ISBN 0 471 84336 9. Wiley, Chichester, 1991. 900 pp. £71.00.
5,555 citations
"Surrogate-based infill optimization..." refers methods in this paper
...Actually, the co-kriging surrogate model [25, 26] is inherently a multifidelity surrogate model that essentially applies a correction to the output of the low-fidelity model....
[...]