Data Mining - Concepts and Techniques.

Deep learning tools have gained tremendous attention in applied machine learning. However such tools for regression and classification do not capture model uncertainty. In comparison, Bayesian models offer a mathematically grounded framework to reason about model uncertainty, but usually come with a prohibitive computational cost. In this paper we develop a new theoretical framework casting dropout training in deep neural networks (NNs) as approximate Bayesian inference in deep Gaussian processes. A direct result of this theory gives us tools to model uncertainty with dropout NNs - extracting information from existing models that has been thrown away so far. This mitigates the problem of representing uncertainty in deep learning without sacrificing either computational complexity or test accuracy. We perform an extensive study of the properties of dropout's uncertainty. Various network architectures and nonlinearities are assessed on tasks of regression and classification, using MNIST as an example. We show a considerable improvement in predictive log-likelihood and RMSE compared to existing state-of-the-art methods, and finish by using dropout's uncertainty in deep reinforcement learning.

/pdf/dropout-as-a-bayesian-approximation-representing-model-94gr4ig3gi.pdf

Dropout as a Bayesian approximation: representing model uncertainty in deep learning

Deep learning tools have gained tremendous attention in applied machine learning. However such tools for regression and classification do not capture model uncertainty. In comparison, Bayesian models offer a mathematically grounded framework to reason about model uncertainty, but usually come with a prohibitive computational cost. In this paper we develop a new theoretical framework casting dropout training in deep neural networks (NNs) as approximate Bayesian inference in deep Gaussian processes. A direct result of this theory gives us tools to model uncertainty with dropout NNs -- extracting information from existing models that has been thrown away so far. This mitigates the problem of representing uncertainty in deep learning without sacrificing either computational complexity or test accuracy. We perform an extensive study of the properties of dropout's uncertainty. Various network architectures and non-linearities are assessed on tasks of regression and classification, using MNIST as an example. We show a considerable improvement in predictive log-likelihood and RMSE compared to existing state-of-the-art methods, and finish by using dropout's uncertainty in deep reinforcement learning.

Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning

The Grid 2: Blueprint for a New Computing Infrastructure

Simulation in the supply chain context: a survey

The United States Department of Defense's High Level Architecture for Modeling and Simulation (HLA) provides a standardized interface for distributed simulations. The recent advent of HLA has greatly increased interest in the use of distributed, interoperable simulation model components. To date, most models using HLA have been developed in conventional high-level languages (primarily C++). This paper presents approaches by which HLA can be used to interconnect distributed model components which are developed using commercially available, off-the-shelf simulation software. The requirements imposed on such simulation software by HLA are discussed, and four approaches for adapting such software for use with HLA are presented. A generalized, model-independent approach which was developed for SLX is presented.

/pdf/internet-based-simulation-using-off-the-shelf-simulation-537bxy8d5x.pdf

Internet-based simulation using off-the-shelf simulation tools and HLA

This paper reports main results of a peer study on future trends in distributed simulation and distributed virtual environments (Strassburger et al. 2008). The peer study was based on the opinions of more than 60 experts which were collected by means of a survey and personal interviews. The survey collected opinions concerning the current state-of-the-art, relevance, and research challenges that must be addressed to advance and strengthen these technologies to a level where they are ready to be applied in day-to-day business in industry. Most important result of this study is the observation that as research areas, both distributed simulation and distributed virtual environments are attributed a high future practical relevance and a high economic potential. At the same time the study shows that the current adoption of these technologies in the industrial sector is rather low. The study analyses reasons for this observation and identifies open research challenges.

/pdf/future-trends-in-distributed-simulation-and-distributed-56fgz2jf7f.pdf

Future trends in distributed simulation and distributed virtual environments: results of a peer study

This paper is intended as an introduction of the challenges that exist in the area of automatic simulation model generation in the production and logistics context. As a work-in-progress paper, it firstly analyzes and classifies previous work; it then introduces the most relevant challenges and lastly presents potential solutions currently being investigated by a PhD thesis.

/pdf/challenges-for-the-automatic-generation-of-simulation-models-1czff7msvg.pdf

Challenges for the automatic generation of simulation models for production systems

The United States Department of Defense's (DoD) High Level Architecture for Modeling and Simulation (HLA) is a mandatory stan dard for military simulations. The situation in the civil simulation co...

/pdf/migration-of-hla-into-civil-domains-solutions-and-prototypes-42l8kwtcgn.pdf

Migration of HLA into Civil Domains: Solutions and Prototypes for Transportation Applications

In this article the technological possibilities offered by the interoperability standard High Level Architecture (HLA) are introduced and discussed. The main focus is hereby on manufacturing applications, but the same approach is applicable to a wide range of other scenarios, e.g. in the area of supply chains, logistics, product simulation etc. Especially for challenging objectives like the digital factory which many enterprises are currently facing, simulation applications are gaining growing importance. While simulations nowadays are often still applied for isolated problems, the consideration of the global context has a growing importance. A solution for solving this problem is offered by the distributed simulation paradigm: simulations are no longer single purpose applications. Rather, individual simulation models can be combined with each other for serving different purposes. Coupled simulations of different parts of a factory can be used to perform global optimizations. The same paradigm can be used for entire supply chains. For applying the distributed simulation paradigm, technological as well as organizational aspects have to be considered. On the technological side, it is necessary to integrate a certain interoperability standard into the tools which need to be coupled with each other. On the organizational level, an enterprise wide process has to be established, which defines how distributed modeling and simulation shall be applied. This article discusses solutions for both issues and illustrates them using a practical application scenario.

Steffen Strassburger

Papers

Internet-based simulation using off-the-shelf simulation tools and HLA

Future trends in distributed simulation and distributed virtual environments: results of a peer study

Challenges for the automatic generation of simulation models for production systems

Migration of HLA into Civil Domains: Solutions and Prototypes for Transportation Applications

Distributed manufacturing simulation as an enabling technology for the digital factory