The purpose of this paper is to provide a comprehensive presentation and interpretation of the Ensemble Kalman Filter (EnKF) and its numerical implementation. The EnKF has a large user group, and numerous publications have discussed applications and theoretical aspects of it. This paper reviews the important results from these studies and also presents new ideas and alternative interpretations which further explain the success of the EnKF. In addition to providing the theoretical framework needed for using the EnKF, there is also a focus on the algorithmic formulation and optimal numerical implementation. A program listing is given for some of the key subroutines. The paper also touches upon specific issues such as the use of nonlinear measurements, in situ profiles of temperature and salinity, and data which are available with high frequency in time. An ensemble based optimal interpolation (EnOI) scheme is presented as a cost-effective approach which may serve as an alternative to the EnKF in some applications. A fairly extensive discussion is devoted to the use of time correlated model errors and the estimation of model bias.

The Ensemble Kalman Filter: Theoretical formulation and practical implementation

Preface. About the Authors. Foreword. Prologue. Part I: Fundamentals. 1. Sampling Plans. 1.1 The 'Curse of Dimensionality' and How to Avoid It. 1.2 Physical versus Computational Experiments. 1.3 Designing Preliminary Experiments (Screening). 1.3.1 Estimating the Distribution of Elementary Effects. 1.4 Designing a Sampling Plan. 1.4.1 Stratification. 1.4.2 Latin Squares and Random Latin Hypercubes. 1.4.3 Space-filling Latin Hypercubes. 1.4.4 Space-filling Subsets. 1.5 A Note on Harmonic Responses. 1.6 Some Pointers for Further Reading. References. 2. Constructing a Surrogate. 2.1 The Modelling Process. 2.1.1 Stage One: Preparing the Data and Choosing a Modelling Approach. 2.1.2 Stage Two: Parameter Estimation and Training. 2.1.3 Stage Three: Model Testing. 2.2 Polynomial Models. 2.2.1 Example One: Aerofoil Drag. 2.2.2 Example Two: a Multimodal Testcase. 2.2.3 What About the k -variable Case? 2.3 Radial Basis Function Models. 2.3.1 Fitting Noise-Free Data. 2.3.2 Radial Basis Function Models of Noisy Data. 2.4 Kriging. 2.4.1 Building the Kriging Model. 2.4.2 Kriging Prediction. 2.5 Support Vector Regression. 2.5.1 The Support Vector Predictor. 2.5.2 The Kernel Trick. 2.5.3 Finding the Support Vectors. 2.5.4 Finding . 2.5.5 Choosing C and epsilon. 2.5.6 Computing epsilon : v -SVR 71. 2.6 The Big(ger) Picture. References. 3. Exploring and Exploiting a Surrogate. 3.1 Searching the Surrogate. 3.2 Infill Criteria. 3.2.1 Prediction Based Exploitation. 3.2.2 Error Based Exploration. 3.2.3 Balanced Exploitation and Exploration. 3.2.4 Conditional Likelihood Approaches. 3.2.5 Other Methods. 3.3 Managing a Surrogate Based Optimization Process. 3.3.1 Which Surrogate for What Use? 3.3.2 How Many Sample Plan and Infill Points? 3.3.3 Convergence Criteria. 3.3.4 Search of the Vibration Isolator Geometry Feasibility Using Kriging Goal Seeking. References. Part II: Advanced Concepts. 4. Visualization. 4.1 Matrices of Contour Plots. 4.2 Nested Dimensions. Reference. 5. Constraints. 5.1 Satisfaction of Constraints by Construction. 5.2 Penalty Functions. 5.3 Example Constrained Problem. 5.3.1 Using a Kriging Model of the Constraint Function. 5.3.2 Using a Kriging Model of the Objective Function. 5.4 Expected Improvement Based Approaches. 5.4.1 Expected Improvement With Simple Penalty Function. 5.4.2 Constrained Expected Improvement. 5.5 Missing Data. 5.5.1 Imputing Data for Infeasible Designs. 5.6 Design of a Helical Compression Spring Using Constrained Expected Improvement. 5.7 Summary. References. 6. Infill Criteria With Noisy Data. 6.1 Regressing Kriging. 6.2 Searching the Regression Model. 6.2.1 Re-Interpolation. 6.2.2 Re-Interpolation With Conditional Likelihood Approaches. 6.3 A Note on Matrix Ill-Conditioning. 6.4 Summary. References. 7. Exploiting Gradient Information. 7.1 Obtaining Gradients. 7.1.1 Finite Differencing. 7.1.2 Complex Step Approximation. 7.1.3 Adjoint Methods and Algorithmic Differentiation. 7.2 Gradient-enhanced Modelling. 7.3 Hessian-enhanced Modelling. 7.4 Summary. References. 8. Multi-fidelity Analysis. 8.1 Co-Kriging. 8.2 One-variable Demonstration. 8.3 Choosing X c and X e . 8.4 Summary. References. 9. Multiple Design Objectives. 9.1 Pareto Optimization. 9.2 Multi-objective Expected Improvement. 9.3 Design of the Nowacki Cantilever Beam Using Multi-objective, Constrained Expected Improvement. 9.4 Design of a Helical Compression Spring Using Multi-objective, Constrained Expected Improvement. 9.5 Summary. References. Appendix: Example Problems. A.1 One-Variable Test Function. A.2 Branin Test Function. A.3 Aerofoil Design. A.4 The Nowacki Beam. A.5 Multi-objective, Constrained Optimal Design of a Helical Compression Spring. A.6 Novel Passive Vibration Isolator Feasibility. References. Index.

Engineering Design via Surrogate Modelling: A Practical Guide

https://eprints.soton.ac.uk/65935/01/Forr_09.pdf

Recent advances in surrogate-based optimization

Engineering Design via Surrogate Modelling

Using surrogate models in evolutionary search provides an efficient means of handling today's complex applications plagued with increasing high-computational needs. Recent surrogate-assisted evolutionary frameworks have relied on the use of a variety of different modeling approaches to approximate the complex problem landscape. From these recent studies, one main research issue is with the choice of modeling scheme used, which has been found to affect the performance of evolutionary search significantly. Given that theoretical knowledge available for making a decision on an approximation model a priori is very much limited, this paper describes a generalization of surrogate-assisted evolutionary frameworks for optimization of problems with objectives and constraints that are computationally expensive to evaluate. The generalized evolutionary framework unifies diverse surrogate models synergistically in the evolutionary search. In particular, it focuses on attaining reliable search performance in the surrogate-assisted evolutionary framework by working on two major issues: 1) to mitigate the 'curse of uncertainty' robustly, and 2) to benefit from the 'bless of uncertainty.' The backbone of the generalized framework is a surrogate-assisted memetic algorithm that conducts simultaneous local searches using ensemble and smoothing surrogate models, with the aims of generating reliable fitness prediction and search improvements simultaneously. Empirical study on commonly used optimization benchmark problems indicates that the generalized framework is capable of attaining reliable, high quality, and efficient performance under a limited computational budget.

/pdf/generalizing-surrogate-assisted-evolutionary-computation-3qbrlsdn60.pdf

Generalizing Surrogate-Assisted Evolutionary Computation

Until recently, optimization was regarded as a discipline of rather theoretical interest, with limited real-life applicability due to the computational or experimental expense involved. Practical multiobjective optimization was considered almost as a utopia even in academic studies due to the multiplication of this expense. This paper discusses the idea of using surrogate models for multiobjective optimization. With recent advances in grid and parallel computing more companies are buying inexpensive computing clusters that work in parallel. This allows, for example, efficient fusion of surrogates and finite element models into a multiobjective optimization cycle. The research presented here demonstrates this idea using several response surface methods on a pre-selected set of test functions. We aim to show that there are number of techniques which can be used to tackle difficult problems and we also demonstrate that a careful choice of response surface methods is important when carrying out surrogate assisted multiobjective search.

/pdf/multi-objective-optimization-using-surrogates-kftllnythh.pdf

Multi-Objective Optimization Using Surrogates

This paper is concerned with the friction surfacing of high-speed steels, BM2, BT15 and ASP30 onto plain carbon steel plate. The events that the matrix and carbides experience as the coating material pass from the coating rod to the substrate, in forming the coating, is described. The coating is observed to harden automatically within a few seconds of being deposited onto the cold substrate. This autohardening is observed to be an inherent feature of the friction surfacing process and the only post-coating heat treatment required is tempering, as with traditionally hardened high-speed steels. The mechanism of autohardening is discussed in terms of the mechtrode/coating/substrate thermal system.

/pdf/on-the-thermo-mechanical-events-during-friction-surfacing-of-4jam46bc3y.pdf

On the thermo-mechanical events during friction surfacing of high speed steels

https://eprints.soton.ac.uk/23248/1/Vout_05.pdf

Weld sequence optimization: The use of surrogate models for solving sequential combinatorial problems

https://eprints.soton.ac.uk/23303/1/Vita_05.pdf

Process parameters selection for friction surfacing applications using intelligent decision support

This paper discusses the procedures for data collection, management and optimisation of the friction surfacing process. Experimental set-up and characteristics of measuring equipment are found to match the requirements for accurate and unbiased data signals. The main friction surfacing parameters are identified and the first stage of the optimisation process is achieved by visually assessing the coatings and introducing the substrate speed vs. force map. The optimum values from this first stage forms a region around the middle of a trapezium-shaped area whose borders are found experimentally. Data collected for the second stage were analysed using the least squares method which were applied to find the coefficients of a second order regression model. Advantages of applying artificial intelligence methods to friction surfacing modelling are also described and the higher accuracy achieved using neural networks demonstrated.

/pdf/an-integrated-approach-to-friction-surfacing-process-1yg4scace8.pdf

Ivan Voutchkov

Papers

Multi-Objective Optimization Using Surrogates

On the thermo-mechanical events during friction surfacing of high speed steels

Weld sequence optimization: The use of surrogate models for solving sequential combinatorial problems

Process parameters selection for friction surfacing applications using intelligent decision support

An integrated approach to friction surfacing process optimisation