THE DESIGN AND ANALYSIS OF EXPERIMENTS. By Oscar Kempthorne. New York, John Wiley and Sons, Inc., 1952. 631 pp. $8.50. This book by a teacher of statistics (as well as a consultant for \"experimenters\") is a comprehensive study of the philosophical background for the statistical design of experiment. It is necessary to have some facility with algebraic notation and manipulation to be able to use the volume intelligently. The problems are presented from the theoretical point of view, without such practical examples as would be helpful for those not acquainted with mathematics. The mathematical justification for the techniques is given. As a somewhat advanced treatment of the design and analysis of experiments, this volume will be interesting and helpful for many who approach statistics theoretically as well as practically. With emphasis on the \"why,\" and with description given broadly, the author relates the subject matter to the general theory of statistics and to the general problem of experimental inference. MARGARET J. ROBERTSON

The Design and Analysis of Experiments

Computation-intensive design problems are becoming increasingly common in manufacturing industries. The computation burden is often caused by expensive analysis and simulation processes in order to reach a comparable level of accuracy as physical testing data. To address such a challenge, approximation or metamodeling techniques are often used. Metamodeling techniques have been developed from many different disciplines including statistics, mathematics, computer science, and various engineering disciplines. These metamodels are initially developed as “surrogates” of the expensive simulation process in order to improve the overall computation efficiency. They are then found to be a valuable tool to support a wide scope of activities in modern engineering design, especially design optimization. This work reviews the state-of-the-art metamodel-based techniques from a practitioner’s perspective according to the role of metamodeling in supporting design optimization, including model approximation, design space exploration, problem formulation, and solving various types of optimization problems. Challenges and future development of metamodeling in support of engineering design is also analyzed and discussed.Copyright © 2006 by ASME

https://www.sfu.ca/~gwa5/index_files/Microsoft%20Word%20-%20ASME-Metamodeling-6-12-06.pdf

Review of Metamodeling Techniques in Support of Engineering Design Optimization

One of the most exciting tools that have entered the material science toolbox in recent years is machine learning. This collection of statistical methods has already proved to be capable of considerably speeding up both fundamental and applied research. At present, we are witnessing an explosion of works that develop and apply machine learning to solid-state systems. We provide a comprehensive overview and analysis of the most recent research in this topic. As a starting point, we introduce machine learning principles, algorithms, descriptors, and databases in materials science. We continue with the description of different machine learning approaches for the discovery of stable materials and the prediction of their crystal structure. Then we discuss research in numerous quantitative structure–property relationships and various approaches for the replacement of first-principle methods by machine learning. We review how active learning and surrogate-based optimization can be applied to improve the rational design process and related examples of applications. Two major questions are always the interpretability of and the physical understanding gained from machine learning models. We consider therefore the different facets of interpretability and their importance in materials science. Finally, we propose solutions and future research paths for various challenges in computational materials science.

/pdf/recent-advances-and-applications-of-machine-learning-in-55gs85xsws.pdf

Recent advances and applications of machine learning in solid-state materials science

Bayesian optimization is an approach to optimizing objective functions that take a long time (minutes or hours) to evaluate. It is best-suited for optimization over continuous domains of less than 20 dimensions, and tolerates stochastic noise in function evaluations. It builds a surrogate for the objective and quantifies the uncertainty in that surrogate using a Bayesian machine learning technique, Gaussian process regression, and then uses an acquisition function defined from this surrogate to decide where to sample. In this tutorial, we describe how Bayesian optimization works, including Gaussian process regression and three common acquisition functions: expected improvement, entropy search, and knowledge gradient. We then discuss more advanced techniques, including running multiple function evaluations in parallel, multi-fidelity and multi-information source optimization, expensive-to-evaluate constraints, random environmental conditions, multi-task Bayesian optimization, and the inclusion of derivative information. We conclude with a discussion of Bayesian optimization software and future research directions in the field. Within our tutorial material we provide a generalization of expected improvement to noisy evaluations, beyond the noise-free setting where it is more commonly applied. This generalization is justified by a formal decision-theoretic argument, standing in contrast to previous ad hoc modifications.

A Tutorial on Bayesian Optimization

The use of kriging models for approximation and metamodel-based design and optimization has been steadily on the rise in the past decade. The widespread usage of kriging models appears to be hampered by (1) the lack of guidance in selecting the appropriate form of the kriging model, (2) computationally efficient algorithms for estimating the model’s parameters, and (3) an effective method to assess the resulting model’s quality. In this paper, we compare (1) Maximum Likelihood Estimation (MLE) and Cross-Validation (CV) parameter estimation methods for selecting a kriging model’s parameters given its form and (2) and an R 2 of prediction and the corrected Akaike Information Criterion for assessing the quality of the created kriging model, permitting the comparison of different forms of a kriging model. These methods are demonstrated with six test problems. Finally, different forms of kriging models are examined to determine if more complex forms are more accurate and easier to fit than simple forms of kriging models for approximating computer models.

http://www2.mae.ufl.edu/haftka/eoed/protected/ASME04.kriging.pdf

Use of Kriging Models to Approximate Deterministic Computer Models

The integration of optimization methodologies with computational analyses/simulations has a profound impact on the product design. Such integration, however, faces multiple challenges. The most eminent challenges arise from high-dimensionality of problems, computationally-expensive analysis/simulation, and unknown function properties (i.e., black-box functions). The merger of these three challenges severely aggravates the difficulty and becomes a major hurdle for design optimization. This paper provides a survey on related modeling and optimization strategies that may help to solve High-dimensional, Expensive (computationally), Black-box (HEB) problems. The survey screens out 207 references including multiple historical reviews on relevant subjects from more than 1,000 papers in a variety of disciplines. This survey has been performed in three areas: strategies tackling high-dimensionality of problems, model approximation techniques, and direct optimization strategies for computationally-expensive black-box functions and promising ideas behind non-gradient optimization algorithms. Major contributions in each area are discussed and presented in an organized manner. The survey exposes that direct modeling and optimization strategies to address HEB problems are scarce and sporadic, partially due to the difficulty of the problem itself. Moreover, it is revealed that current modeling research tends to focus on sampling and modeling techniques themselves and neglect studying and taking the advantages of characteristics of the underlying expensive functions. Based on the survey results, two promising approaches are identified to solve HEB problems. Directions for future research are also discussed.

/pdf/survey-of-modeling-and-optimization-strategies-to-solve-high-1tg25zsec9.pdf

Survey of modeling and optimization strategies to solve high-dimensional design problems with computationally-expensive black-box functions

The presence of black-box functions in engineering design, which are usually computation-intensive, demands efficient global optimization methods. This article proposes a new global optimization method for black-box functions. The global optimization method is based on a novel mode-pursuing sampling method that systematically generates more sample points in the neighborhood of the function mode while statistically covering the entire search space. Quadratic regression is performed to detect the region containing the global optimum. The sampling and detection process iterates until the global optimum is obtained. Through intensive testing, this method is found to be effective, efficient, robust, and applicable to both continuous and discontinuous functions. It supports simultaneous computation and applies to both unconstrained and constrained optimization problems. Because it does not call any existing global optimization tool, it can be used as a standalone global optimization method for inexpensive probl...

/pdf/mode-pursuing-sampling-method-for-global-optimization-on-5bq995dofi.pdf

Mode-pursuing sampling method for global optimization on expensive black-box functions

Reliable design space and complete single-loop reliability-based design optimization

Computational tools such as finite element analysis and simulation are widely used in engineering, but they are mostly used for design analysis and validation. If these tools can be integrated for design optimization, it will undoubtedly enhance a manufacturer's competitiveness. Such integration, however, faces three main challenges: (1) high computational expense of simulation, (2) the simulation process being a black-box function, and (3) design problems being high dimensional. In the past two decades, metamodeling has been intensively developed to deal with expensive black-box functions, and has achieved success for low dimensional design problems. But when high dimensionality is also present in design, which is often found in practice, there lacks of a practical method to deal with the so-called high dimensional, expensive, and black-box (HEB) problems. This paper proposes the first metamodel of its kind to tackle the HEB problem. This paper integrates the radial basis function with high dimensional model representation into a new model, RBF-HDMR. The developed RBF-HDMR model offers an explicit function expression, and can reveal (1 ) the contribution of each design variable, (2) inherent linearity/nonlinearity with respect to input variables, and (3) correlation relationships among input variables. An accompanying algorithm to construct the RBF - HDMR has also been developed. The model and the algorithm fundamentally change the exponentially growing computation cost to be polynomial. Testing and comparison confirm the efficiency and capability of RBF-HDMR for HEB problems.

/pdf/metamodeling-for-high-dimensional-simulation-based-design-5da3ycvf7a.pdf

Songqing Shan

Papers

Review of Metamodeling Techniques in Support of Engineering Design Optimization

Survey of modeling and optimization strategies to solve high-dimensional design problems with computationally-expensive black-box functions

Mode-pursuing sampling method for global optimization on expensive black-box functions

Reliable design space and complete single-loop reliability-based design optimization

Metamodeling for High Dimensional Simulation-Based Design Problems