https://www2.econ.iastate.edu/tesfatsi/DeepLearningInNeuralNetworksOverview.JSchmidhuber2015.pdf

Deep learning in neural networks

Preface 1. Introduction Overview A Brief History of LMIs in Control Theory Notes on the Style of the Book Origin of the Book 2. Some Standard Problems Involving LMIs. Linear Matrix Inequalities Some Standard Problems Ellipsoid Algorithm Interior-Point Methods Strict and Nonstrict LMIs Miscellaneous Results on Matrix Inequalities Some LMI Problems with Analytic Solutions 3. Some Matrix Problems. Minimizing Condition Number by Scaling Minimizing Condition Number of a Positive-Definite Matrix Minimizing Norm by Scaling Rescaling a Matrix Positive-Definite Matrix Completion Problems Quadratic Approximation of a Polytopic Norm Ellipsoidal Approximation 4. Linear Differential Inclusions. Differential Inclusions Some Specific LDIs Nonlinear System Analysis via LDIs 5. Analysis of LDIs: State Properties. Quadratic Stability Invariant Ellipsoids 6. Analysis of LDIs: Input/Output Properties. Input-to-State Properties State-to-Output Properties Input-to-Output Properties 7. State-Feedback Synthesis for LDIs. Static State-Feedback Controllers State Properties Input-to-State Properties State-to-Output Properties Input-to-Output Properties Observer-Based Controllers for Nonlinear Systems 8. Lure and Multiplier Methods. Analysis of Lure Systems Integral Quadratic Constraints Multipliers for Systems with Unknown Parameters 9. Systems with Multiplicative Noise. Analysis of Systems with Multiplicative Noise State-Feedback Synthesis 10. Miscellaneous Problems. Optimization over an Affine Family of Linear Systems Analysis of Systems with LTI Perturbations Positive Orthant Stabilizability Linear Systems with Delays Interpolation Problems The Inverse Problem of Optimal Control System Realization Problems Multi-Criterion LQG Nonconvex Multi-Criterion Quadratic Problems Notation List of Acronyms Bibliography Index.

/pdf/linear-matrix-inequalities-in-system-and-control-theory-2q5cf0pqm2.pdf

Linear Matrix Inequalities in System and Control Theory

Multiple, often conflicting objectives arise naturally in most real-world optimization scenarios. As evolutionary algorithms possess several characteristics due to which they are well suited to this type of problem, evolution-based methods have been used for multiobjective optimization for more than a decade. Meanwhile evolutionary multiobjective optimization has become established as a separate subdiscipline combining the fields of evolutionary computation and classical multiple criteria decision making. In this paper, the basic principles of evolutionary multiobjective optimization are discussed from an algorithm design perspective. The focus is on the major issues such as fitness assignment, diversity preservation, and elitism in general rather than on particular algorithms. Different techniques to implement these strongly related concepts will be discussed, and further important aspects such as constraint handling and preference articulation are treated as well. Finally, two applications will presented and some recent trends in the field will be outlined.

Evolutionary Algorithms for Multiobjective Optimization: Methods and Applications

We give two general convergence proofs for random search algorithms. We review the literature and show how our results extend those available for specific variants of the conceptual algorithm studied here. We then exploit the convergence results to examine convergence rates and to actually design implementable methods. Finally we report on some computational experience.

/pdf/minimization-by-random-search-techniques-32qtk73kov.pdf

Minimization by Random Search Techniques

An artificial neural network (ANN) is a flexible mathematical structure which is capable of identifying complex nonlinear relationships between input and output data sets. ANN models have been found useful and efficient, particularly in problems for which the characteristics of the processes are difficult to describe using physical equations. This study presents a new procedure (entitled linear least squares simplex, or LLSSIM) for identifying the structure and parameters of three-layer feed forward ANN models and demonstrates the potential of such models for simulating the nonlinear hydrologic behavior of watersheds. The nonlinear ANN model approach is shown to provide a better representation of the rainfall-runoff relationship of the medium-size Leaf River basin near Collins, Mississippi, than the linear ARMAX (autoregressive moving average with exogenous inputs) time series approach or the conceptual SAC-SMA (Sacramento soil moisture accounting) model. Because the ANN approach presented here does not provide models that have physically realistic components and parameters, it is by no means a substitute for conceptual watershed modeling. However, the ANN approach does provide a viable and effective alternative to the ARMAX time series approach for developing input-output simulation and forecasting models in situations that do not require modeling of the internal structure of the watershed.

Artificial Neural Network Modeling of the Rainfall‐Runoff Process

Theory of Multiobjective Optimization

A heuristic self-organization method for constructing a nonlinear river flow prediction model from the available data such as river flow and areal mean precipitation is presented. Our algorithm, the improved version of the GMDH proposed by A. G. Ivakhnenko, is useful for the prediction of complex nonlinear systems with a large number of variables and with a small amount of available input-output data. The efficiency and usefulness of the proposed sequential prediction algorithm are shown by the use of a simulation model. This algorithm is applied to the flow prediction of the Karasu River in Japan. Numerical comparisons are performed between the prediction model by "sequential GMDH" and by the elaborate hydrologic methods, and we show that there are improvements in the newly introduced prediction algorithm for real-time computation.

Sequential GMDH Algorithm and Its Application to River Flow Prediction

/pdf/discrete-time-markovian-decision-processes-with-incomplete-551j7t7wmg.pdf

Discrete-Time Markovian Decision Processes with Incomplete State Observation

This paper deals with the state estimation problem for linear discrete time systems with the interrupted observation mechanisms which can be characterized by the independent binary sequence taking on the values of 0 or 1. On the basis of the Bayesian approach, the approximate minimum variance estimator is derived for the case when at any time the probability of occurrence of interruption is known a priori. An adaptive estimator algorithm is also established when the probability of interruption is unknown but fixed throughout the time interval of interest. Unlike the usual Kalman filter algorithm, all the estimators derived here are nonlinear with respect to observations and the associated covariance equations are related to the actual observations. Digital simulation studies are demonstrated to compare the performance of the approximate nonlinear estimator presented here with that of the best linear estimator due to Nahi (1969) .

Yoshikazu Sawaragi

Papers

Theory of Multiobjective Optimization

Sequential GMDH Algorithm and Its Application to River Flow Prediction

Discrete-Time Markovian Decision Processes with Incomplete State Observation

Sequential state estimation with interrupted observation

A modified convergence theorem for a random optimization method