Computation-intensive design problems are becoming increasingly common in manufacturing industries. The computation burden is often caused by expensive analysis and simulation processes in order to reach a comparable level of accuracy as physical testing data. To address such a challenge, approximation or metamodeling techniques are often used. Metamodeling techniques have been developed from many different disciplines including statistics, mathematics, computer science, and various engineering disciplines. These metamodels are initially developed as “surrogates” of the expensive simulation process in order to improve the overall computation efficiency. They are then found to be a valuable tool to support a wide scope of activities in modern engineering design, especially design optimization. This work reviews the state-of-the-art metamodel-based techniques from a practitioner’s perspective according to the role of metamodeling in supporting design optimization, including model approximation, design space exploration, problem formulation, and solving various types of optimization problems. Challenges and future development of metamodeling in support of engineering design is also analyzed and discussed.Copyright © 2006 by ASME

https://www.sfu.ca/~gwa5/index_files/Microsoft%20Word%20-%20ASME-Metamodeling-6-12-06.pdf

Review of Metamodeling Techniques in Support of Engineering Design Optimization

Many engineering applications are characterized by implicit response functions that are expensive to evaluate and sometimes nonlinear in their behavior, making reliability analysis difficult. This paper develops an efficient reliability analysis method that accurately characterizes the limit state throughout the random variable space. The method begins with a Gaussian process model built from a very small number of samples, and then adaptively chooses where to generate subsequent samples to ensure that the model is accurate in the vicinity of the limit state. The resulting Gaussian process model is then sampled using multimodal adaptive importance sampling to calculate the probability of exceeding (or failing to exceed) the response level of interest. By locating multiple points on or near the limit state, more complex and nonlinear limit states can be modeled, leading to more accurate probability integration. By concentrating the samples in the area where accuracy is important (i.e., in the vicinity of the limit state), only a small number of true function evaluations are required to build a quality surrogate model. The resulting method is both accurate for any arbitrarily shaped limit state and computationally efficient even for expensive response functions. This new method is applied to a collection of example problems including one that analyzes the reliability of a microelectromechanical system device that current available methods have difficulty solving either accurately or efficiently.

Efficient Global Reliability Analysis for Nonlinear Implicit Performance Functions

The DAKOTA (Design Analysis Kit for Optimization and Terascale Applications) toolkit provides a flexible and extensible interface between simulation codes and iterative analysis methods. DAKOTA contains algorithms for optimization with gradient and nongradient-based methods; uncertainty quantification with sampling, reliability, and stochastic finite element methods; parameter estimation with nonlinear least squares methods; and sensitivity/variance analysis with design of experiments and parameter study methods. These capabilities may be used on their own or as components within advanced strategies such as surrogate-based optimization, mixed integer nonlinear programming, or optimization under uncertainty. By employing object-oriented design to implement abstractions of the key components required for iterative systems analyses, the DAKOTA toolkit provides a flexible and extensible problem-solving environment for design and performance analysis of computational models on high performance computers. This report serves as a reference manual for the commands specification for the DAKOTA software, providing input overviews, option descriptions, and example specifications. DAKOTA Version 5.0 Reference Manual generated on May 7, 2010

/pdf/dakota-a-multilevel-parallel-object-oriented-framework-for-52psgais7x.pdf

DAKOTA : a multilevel parallel object-oriented framework for design optimization, parameter estimation, uncertainty quantification, and sensitivity analysis. Version 5.0, user's manual.

This paper proposes a new method that extends the efficient global optimization to address stochastic black-box systems. The method is based on a kriging meta-model that provides a global prediction of the objective values and a measure of prediction uncertainty at every point. The criterion for the infill sample selection is an augmented expected improvement function with desirable properties for stochastic responses. The method is empirically compared with the revised simplex search, the simultaneous perturbation stochastic approximation, and the DIRECT methods using six test problems from the literature. An application case study on an inventory system is also documented. The results suggest that the proposed method has excellent consistency and efficiency in finding global optimal solutions, and is particularly useful for expensive systems.

Global Optimization of Stochastic Black-Box Systems via Sequential Kriging Meta-Models

The integration of optimization methodologies with computational analyses/simulations has a profound impact on the product design. Such integration, however, faces multiple challenges. The most eminent challenges arise from high-dimensionality of problems, computationally-expensive analysis/simulation, and unknown function properties (i.e., black-box functions). The merger of these three challenges severely aggravates the difficulty and becomes a major hurdle for design optimization. This paper provides a survey on related modeling and optimization strategies that may help to solve High-dimensional, Expensive (computationally), Black-box (HEB) problems. The survey screens out 207 references including multiple historical reviews on relevant subjects from more than 1,000 papers in a variety of disciplines. This survey has been performed in three areas: strategies tackling high-dimensionality of problems, model approximation techniques, and direct optimization strategies for computationally-expensive black-box functions and promising ideas behind non-gradient optimization algorithms. Major contributions in each area are discussed and presented in an organized manner. The survey exposes that direct modeling and optimization strategies to address HEB problems are scarce and sporadic, partially due to the difficulty of the problem itself. Moreover, it is revealed that current modeling research tends to focus on sampling and modeling techniques themselves and neglect studying and taking the advantages of characteristics of the underlying expensive functions. Based on the survey results, two promising approaches are identified to solve HEB problems. Directions for future research are also discussed.

/pdf/survey-of-modeling-and-optimization-strategies-to-solve-high-1tg25zsec9.pdf

Survey of modeling and optimization strategies to solve high-dimensional design problems with computationally-expensive black-box functions

The dimensionality and complexity of typical multidisciplinary systems hinders the use of formal optimization techniques in application to this class of problems. The use of approximations to represent the system design metrics and constraints has become vital for achieving good performance in many multidisciplinary design optimization (MDO) algorithms. This paper reports recent research efforts on the use of variable fidelity response surface approximations (RSA) to drive the convergence of MDO problems using a trust region model management algorithm. The present study focuses on a comparative study of different response sampling strategies based on design of experiment (DOE) approaches within the disciplines to generate the zero order data to build the RSAs. Two MDO test problems that have complex coupling between disciplines are used to benchmark the performance of each sampling strategy. The results show that these types of variable fidelity RSAs can be effectively managed by the trust region model management strategy to drive convergence of MDO problems. It is observed that the efficiency of the optimization algorithm depends on the sampling strategy used. A comparison of the DOE approaches with those obtained using a optimization based sampling strategy (i.e. concurrent subspace optimization --- CSSO) shows the DOE methodologies to be competitive with the CSSO based sampling methodology in some cases. However, the CSSO based sampling strategy was found to be, in general, more efficient in driving the optimization.

Sequential approximate optimization using variable fidelity response surface approximations

Reliability methods are probabilistic algorithms for quantifying the effect of simulation input uncertainties on response metrics of interest. In particular, they compute approximate response function distribution statistics (probability, reliability and response levels) based on specified input random variable probability distributions. In this paper, a number of algorithmic variations are explored for both the forward reliability analysis of computing probabilities for specified response levels (the reliability index approach (RIA)) and the inverse reliability analysis of computing response levels for specified probabilities (the performance measure approach (PMA)). These variations include limit state linearizations, probability integrations, warm starting and optimization algorithm selections. The resulting RIA/PMA reliability algorithms for uncertainty quantification are then employed within bi-level and sequential reliability-based design optimization approaches. Relative performance of these uncert...

Investigation of reliability method formulations in DAKOTA/UQ

Sequential Approximate Optimization (SAO) is a class of methods available for the multidisciplinary design optimization (MDO) of complex systems that are composed of several disciplines coupled together. One of the approaches used for SAO, is based on a quadratic response surface approximation, where zero and first order information are required. In these methods, designers must generate and query a database of order O(n2) in order to compute the second order terms of the quadratic response surface approximation. As the number of design variables grows, the computational cost of generating the required database becomes a concern. In this paper, we present an new approach in which we require just O(ri) parameters for constructing a second order approximation. This is accomplished by transforming the matrix of second order terms into the canonical form. The method periodically requires an order O(n2) update of the second order approximation to maintain accuracy. Results show that the proposed approach is practical and convenient for engineering design problems by dramatically reducing the total number of calls to the simulation tools.

Adaptive Experimental Design for Construction of Response Surface Approximations

The use of optimization in a simulation-based design environment has become a common trend in industry today. Computer simulation tools are commonplace in many engineering disciplines, providing the designers with tools to evaluate a design’s performance without building a physical prototype. This has triggered the development of optimization techniques suitable for dealing with such simulations. One of these approaches is known as sequential approximate optimization. In sequential approximate minimization a sequence of optimizations are performed over local response surface approximations of the system. This paper details the development of an interior-point approach for trust-region-managed sequential approximate optimization. The interior-point approach will ensure that approximate feasibility is maintained throughout the optimization process. This facilitates the delivery of a usable design at each iteration when subject to reduced design cycle time constraints. In order to deal with infeasible starting points, homotopy methods are used to relax constraints and push designs toward feasibility. Results of application studies are presented, illustrating the applicability of the proposed algorithm.

An interior-point sequential approximate optimization methodology

Purpose – To reduce the computational complexity per step from O(n2) to O(n) for optimization based on quadratic surrogates, where n is the number of design variables.Design/methodology/approach – Applying nonlinear optimization strategies directly to complex multidisciplinary systems can be prohibitively expensive when the complexity of the simulation codes is large. Increasingly, response surface approximations (RSAs), and specifically quadratic approximations, are being integrated with nonlinear optimizers in order to reduce the CPU time required for the optimization of complex multidisciplinary systems. For evaluation by the optimizer, RSAs provide a computationally inexpensive lower fidelity representation of the system performance. The curse of dimensionality is a major drawback in the implementation of these approximations as the amount of required data grows quadratically with the number n of design variables in the problem. In this paper a novel technique to reduce the magnitude of the sampling f...

/pdf/reduced-sampling-for-construction-of-quadratic-response-1m1h2untzw.pdf

Victor Perez

Papers

Sequential approximate optimization using variable fidelity response surface approximations

Investigation of reliability method formulations in DAKOTA/UQ

Adaptive Experimental Design for Construction of Response Surface Approximations

An interior-point sequential approximate optimization methodology

Reduced Sampling for Construction of Quadratic Response Surface Approximations Using Adaptive Experimental Design