When building a unified vision system or gradually adding new capabilities to a system, the usual assumption is that training data for all tasks is always available. However, as the number of tasks grows, storing and retraining on such data becomes infeasible. A new problem arises where we add new capabilities to a Convolutional Neural Network (CNN), but the training data for its existing capabilities are unavailable. We propose our Learning without Forgetting method, which uses only new task data to train the network while preserving the original capabilities. Our method performs favorably compared to commonly used feature extraction and fine-tuning adaption techniques and performs similarly to multitask learning that uses original task data we assume unavailable. A more surprising observation is that Learning without Forgetting may be able to replace fine-tuning with similar old and new task datasets for improved new task performance.

Learning without Forgetting

Statistical Models in Engineering

Probability, Reliability and Statistical Methods in Engineering Design

Optimization for structural crashworthiness and energy absorption has become an important topic of research attributable to its proven benefits to public safety and social economy. This paper provides a comprehensive review of the important studies on design optimization for structural crashworthiness and energy absorption. First, the design criteria used in crashworthiness and energy absorption are reviewed and the surrogate modeling to evaluate these criteria is discussed. Second, multiobjective optimization, optimization under uncertainties and topology optimization are reviewed from concepts, algorithms to applications in relation to crashworthiness. Third, the crashworthy structures are summarized, from generically novel structural configurations to industrial applications. Finally, some conclusions and recommendations are provided to enable academia and industry to become more aware of the available capabilities and recent developments in design optimization for structural crashworthiness and energy absorption.

On design optimization for structural crashworthiness and its state of the art

https://link.springer.com/content/pdf/10.1057%2Fjors.1994..pdf

Discrete Event Systems: Sensitivity Analysis and Stochastic Optimization by the Score Function Method

The reliability-based design optimization (RBDO) using performance measure approach for problems with correlated input variables requires a transformation from the correlated input random variables into independent standard normal variables. For the transformation with correlated input variables, the two most representative transformations, the Rosenblatt and Nataf transformations, are investigated. The Rosenblatt transformation requires a joint cumulative distribution function (CDF). Thus, the Rosenblatt transformation can be used only if the joint CDF is given or input variables are independent. In the Nataf transformation, the joint CDF is approximated using the Gaussian copula, marginal CDFs, and covariance of the input correlated variables. Using the generated CDF, the correlated input variables are transformed into correlated normal variables and then the correlated normal variables are transformed into independent standard normal variables through a linear transformation. Thus, the Nataf transformation can accurately estimates joint normal and some lognormal CDFs of the input variable that cover broad engineering applications. This paper develops a PMA-based RBDO method for problems with correlated random input variables using the Gaussian copula. Several numerical examples show that the correlated random input variables significantly affect RBDO results.

Reliability-based design optimization of problems with correlated input variables using a Gaussian Copula

This study presents a methodology for computing stochastic sensitivities with respect to the design variables, which are the mean values of the input correlated random variables. Assuming that an accurate surrogate model is available, the proposed method calculates the component reliability, system reliability, or statistical moments and their sensitivities by applying Monte Carlo simulation to the accurate surrogate model. Since the surrogate model is used, the computational cost for the stochastic sensitivity analysis is affordable compared with the use of actual models. The copula is used to model the joint distribution of the correlated input random variables, and the score function is used to derive the stochastic sensitivities of reliability or statistical moments for the correlated random variables. An important merit of the proposed method is that it does not require the gradients of performance functions, which are known to be erroneous when obtained from the surrogate model, or the transformation from X-space to U-space for reliability analysis. Since no transformation is required and the reliability or statistical moment is calculated in X-space, there is no approximation or restriction in calculating the sensitivities of the reliability or statistical moment. Numerical results indicate that the proposed method can estimate the sensitivities of the reliability or statistical moments very accurately, even when the input random variables are correlated.

Sampling-Based Stochastic Sensitivity Analysis Using Score Functions for RBDO Problems with Correlated Random Variables

In RBDO, input uncertainty models such as marginal and joint cumulative distribution functions (CDFs) need to be used. However, only limited data exists in industry applications. Thus, identification of the input uncertainty model is challenging especially when input variables are correlated. Since input random variables, such as fatigue material properties, are correlated in many industrial problems, the joint CDF of correlated input variables needs to be correctly identified from given data. In this paper, a Bayesian method is proposed to identify the marginal and joint CDFs from given data where a copula, which only requires marginal CDFs and correlation parameters, is used to model the joint CDF of input variables. Using simulated data sets, performance of the Bayesian method is tested for different numbers of samples and is compared with the goodness-of-fit (GOF) test. Two examples are used to demonstrate how the Bayesian method is used to identify correct marginal CDFs and copula.

Identification of marginal and joint CDFs using Bayesian method for RBDO

This paper proposes a novel second-order reliability method (SORM) using non-central or general chi-squared distribution to improve the accuracy of reliability analysis in existing SORM. Conventional SORM contains three types of errors: (1) error due to approximating a general nonlinear limit state function by a quadratic function at most probable point (MPP) in the standard normal U-space, (2) error due to approximating the quadratic function in U-space by a hyperbolic surface, and (3) error due to calculation of the probability of failure after making the previous two approximations. The proposed method contains the first type of error only which is essential to SORM and thus cannot be improved. However, the proposed method avoids the other two errors by describing the quadratic failure surface with the linear combination of non-central chi-square variables and using the linear combination for the probability of failure estimation. Two approaches for the proposed SORM are suggested in the paper. The first approach directly calculates the probability of failure using numerical integration of the joint probability density function (PDF) over the linear failure surface and the second approach uses the cumulative distribution function (CDF) of the linear failure surface for the calculation of the probability of failure. The proposed method is compared with first-order reliability method (FORM), conventional SORM, and Monte Carlo simulation (MCS) results in terms of accuracy. Since it contains fewer approximations, the proposed method shows more accurate reliability analysis results than existing SORM without sacrificing efficiency.Copyright © 2012 by ASME

A Novel Second-Order Reliability Method (SORM) Using Noncentral or Generalized Chi-Squared Distributions

A smart ship collects various data with large volume, such as voyage, machinery, and weather data. Thus, big data analysis for smart ships is an important technology that can be widely applied to improve ship maintenance, operational efficiency, and equipment life management. In this study, an accurate regression model for the fuel consumption of the main engine by using an artificial neural network (ANN) was proposed by big data analysis including data collection, clustering, compression, and expansion. To obtain an accurate regression model, various numbers of hidden layers and neurons and different types of activation functions were tested in the ANN, and their effects on the accuracy and efficiency of the regression analysis were studied. The proposed regression model using ANN is a more accurate and efficient model to predict the fuel consumption of the main engine than polynomial regression and support vector machine.

Yoojeong Noh

Papers

Reliability-based design optimization of problems with correlated input variables using a Gaussian Copula

Sampling-Based Stochastic Sensitivity Analysis Using Score Functions for RBDO Problems with Correlated Random Variables

Identification of marginal and joint CDFs using Bayesian method for RBDO

A Novel Second-Order Reliability Method (SORM) Using Noncentral or Generalized Chi-Squared Distributions

Prediction of ship fuel consumption by using an artificial neural network