Convolutional networks are at the core of most state of-the-art computer vision solutions for a wide variety of tasks. Since 2014 very deep convolutional networks started to become mainstream, yielding substantial gains in various benchmarks. Although increased model size and computational cost tend to translate to immediate quality gains for most tasks (as long as enough labeled data is provided for training), computational efficiency and low parameter count are still enabling factors for various use cases such as mobile vision and big-data scenarios. Here we are exploring ways to scale up networks in ways that aim at utilizing the added computation as efficiently as possible by suitably factorized convolutions and aggressive regularization. We benchmark our methods on the ILSVRC 2012 classification challenge validation set demonstrate substantial gains over the state of the art: 21:2% top-1 and 5:6% top-5 error for single frame evaluation using a network with a computational cost of 5 billion multiply-adds per inference and with using less than 25 million parameters. With an ensemble of 4 models and multi-crop evaluation, we report 3:5% top-5 error and 17:3% top-1 error on the validation set and 3:6% top-5 error on the official test set.

Rethinking the Inception Architecture for Computer Vision

Convolutional networks are at the core of most state-of-the-art computer vision solutions for a wide variety of tasks. Since 2014 very deep convolutional networks started to become mainstream, yielding substantial gains in various benchmarks. Although increased model size and computational cost tend to translate to immediate quality gains for most tasks (as long as enough labeled data is provided for training), computational efficiency and low parameter count are still enabling factors for various use cases such as mobile vision and big-data scenarios. Here we explore ways to scale up networks in ways that aim at utilizing the added computation as efficiently as possible by suitably factorized convolutions and aggressive regularization. We benchmark our methods on the ILSVRC 2012 classification challenge validation set demonstrate substantial gains over the state of the art: 21.2% top-1 and 5.6% top-5 error for single frame evaluation using a network with a computational cost of 5 billion multiply-adds per inference and with using less than 25 million parameters. With an ensemble of 4 models and multi-crop evaluation, we report 3.5% top-5 error on the validation set (3.6% error on the test set) and 17.3% top-1 error on the validation set.

Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations

From the Publisher:
Fuzzy Modeling for Control addresses fuzzy modeling from the systems and control engineering point of view. It focuses on the selection of appropriate model structures, on the acquisition of dynamic fuzzy models from process measurements (fuzzy identification), and on the design of nonlinear controllers based on fuzzy models. The main features of the presented techniques are illustrated by means of simple examples. In addition, three real-world applications are described. Finally, software tools for building fuzzy models from measurements are available from the author.

Fuzzy Modeling for Control

A hybrid neural network-first principles modeling scheme is developed and used to model a fedbatch bioreactor. The hybrid model combines a partial first principles model, which incorporates the available prior knowledge about the process being modeled, with a neural network which serves as an estimator of unmeasuredprocess parameters that are difficult to model from first principles. This hybrid model has better properties than standard “black-box” neural network models in that it is able to interpolate and extrapolate much more accurately, is easier to analyze and interpret, and requires significantly fewer training examples. Two alternative state and parameter estimation strategies, extended Kalman filtering and NLP optimization, are also considered. When no a priori known model of the unobserved process parameters is available, the hybrid network model gives better estimates of the parameters, when compared to these methods. By providing a model of these unmeasured parameters, the hybrid network can also make predictions and hence can be used for process optimization. These results apply both when full and partial state measurements are available, but in the latter case a state reconstruction method must be used for the first principles component of the hybrid model.

/pdf/a-hybrid-neural-network-first-principles-approach-to-process-sqgzbd4a3l.pdf

A hybrid neural network‐first principles approach to process modeling

The use of artificial neural networks in model based control, both as process models and as controllers, is investigated: two nonlinear model based control strategies, internal model control (IMC) and multistep predictive control (MPC), are applied to the control of a nonlinear SISO exothermic continuous stirred tank reactor (CSTR)

Direct and indirect model based control using artificial neural networks

A comparison of two nonparametric estimation schemes: MARS and neural networks

An algorithm is developed for training feedforward neural networks that uses singular value decomposition (SVD) to identify and eliminate redundant hidden nodes. Minimizing redundancy gives smaller networks, producing models that generalize better and thus eliminate the need of using cross-validation to avoid overfitting. The method is demonstrated by modeling a chemical reactor. >

SVD-NET: an algorithm that automatically selects network structure

The ramifications of incorporating neural networks into two model-based control architectures, namely internal model control (IMC) and model predictive control (MPC), are considered. The development of a neural network analog to the conventional IMC design is described, the controller behavior is discussed, and control architectures necessary to improve controller performance are presented. The performance of the neural network controller under less restrictive assumptions is examined, and a neural network analog to the conventional MPC design is developed and tested. An IMC-type neural network controller in which the process model replaced by a neural network gives very good performance, even when only partial state data are available, also gives excellent performance. These results indicate that neural networks can learn accurate models and give good nonlinear control when model equations are not known. Suggestions for improving performance are presented. >

Dimitris C. Psichogios

Papers

A hybrid neural network‐first principles approach to process modeling

Direct and indirect model based control using artificial neural networks

A comparison of two nonparametric estimation schemes: MARS and neural networks

SVD-NET: an algorithm that automatically selects network structure

Nonlinear internal model control and model predictive control using neural networks