Monte Carlo methods are revolutionizing the on-line analysis of data in fields as diverse as financial modeling, target tracking and computer vision. These methods, appearing under the names of bootstrap filters, condensation, optimal Monte Carlo filters, particle filters and survival of the fittest, have made it possible to solve numerically many complex, non-standard problems that were previously intractable. This book presents the first comprehensive treatment of these techniques, including convergence results and applications to tracking, guidance, automated target recognition, aircraft navigation, robot navigation, econometrics, financial modeling, neural networks, optimal control, optimal filtering, communications, reinforcement learning, signal enhancement, model averaging and selection, computer vision, semiconductor design, population biology, dynamic Bayesian networks, and time series analysis. This will be of great value to students, researchers and practitioners, who have some basic knowledge of probability. Arnaud Doucet received the Ph. D. degree from the University of Paris-XI Orsay in 1997. From 1998 to 2000, he conducted research at the Signal Processing Group of Cambridge University, UK. He is currently an assistant professor at the Department of Electrical Engineering of Melbourne University, Australia. His research interests include Bayesian statistics, dynamic models and Monte Carlo methods. Nando de Freitas obtained a Ph.D. degree in information engineering from Cambridge University in 1999. He is presently a research associate with the artificial intelligence group of the University of California at Berkeley. His main research interests are in Bayesian statistics and the application of on-line and batch Monte Carlo methods to machine learning. Neil Gordon obtained a Ph.D. in Statistics from Imperial College, University of London in 1993. He is with the Pattern and Information Processing group at the Defence Evaluation and Research Agency in the United Kingdom. His research interests are in time series, statistical data analysis, and pattern recognition with a particular emphasis on target tracking and missile guidance.

/pdf/sequential-monte-carlo-methods-in-practice-3kfxeebn8e.pdf

Sequential Monte Carlo methods in practice

Deep learning (DL) is playing an increasingly important role in our lives. It has already made a huge impact in areas, such as cancer diagnosis, precision medicine, self-driving cars, predictive forecasting, and speech recognition. The painstakingly handcrafted feature extractors used in traditional learning, classification, and pattern recognition systems are not scalable for large-sized data sets. In many cases, depending on the problem complexity, DL can also overcome the limitations of earlier shallow networks that prevented efficient training and abstractions of hierarchical representations of multi-dimensional training data. Deep neural network (DNN) uses multiple (deep) layers of units with highly optimized algorithms and architectures. This paper reviews several optimization methods to improve the accuracy of the training and to reduce training time. We delve into the math behind training algorithms used in recent deep networks. We describe current shortcomings, enhancements, and implementations. The review also covers different types of deep architectures, such as deep convolution networks, deep residual networks, recurrent neural networks, reinforcement learning, variational autoencoders, and others.

/pdf/review-of-deep-learning-algorithms-and-architectures-41h40bgta6.pdf

Review of Deep Learning Algorithms and Architectures

An adaptive control system uses a neural network to provide adaptive control when the plant is operating within a normal operating range, but shifts to other types of control as the plant operating conditions move outside of the normal operating range. The controller uses a structure which allows the neural network parameters to be determined from minimal information about plant structure and the neural network is trained on-line during normal plant operation. The resulting system can be proven to be stable over all possible conditions. Further, with the inventive techniques, the tracking accuracy can be controlled by appropriate network design.

Stable adaptive neural network controller

Natural DNA can encode complexity on an enormous scale. Researchers are attempting to achieve the same representational efficiency in computers by implementing developmental encodings, i.e. encodings that map the genotype to the phenotype through a process of growth from a small starting point to a mature form. A major challenge in in this effort is to find the right level of abstraction of biological development to capture its essential properties without introducing unnecessary inefficiencies. In this paper, a novel abstraction of natural development, called Compositional Pattern Producing Networks (CPPNs), is proposed. Unlike currently accepted abstractions such as iterative rewrite systems and cellular growth simulations, CPPNs map to the phenotype without local interaction, that is, each individual component of the phenotype is determined independently of every other component. Results produced with CPPNs through interactive evolution of two-dimensional images show that such an encoding can nevertheless produce structural motifs often attributed to more conventional developmental abstractions, suggesting that local interaction may not be essential to the desirable properties of natural encoding in the way that is usually assumed.

/pdf/compositional-pattern-producing-networks-a-novel-abstraction-49f5ds9t48.pdf

Compositional pattern producing networks: A novel abstraction of development

Within this text neural networks are considered as massively interconnected nonlinear adaptive filters.

Recurrent Neural Networks for Prediction

We generalize a result of Gao and Xu [4] concerning the approximation of functions of bounded variation by linear combinations of a fixed sigmoidal function to the class of functions of bounded φ-variation (Theorem 2.7). Also, in the case of one variable, [1: Proposition 1] is improved. Our proofs are similar to that of [4].

https://www.ems-ph.org/journals/show_pdf.php?issn=0232-2064&vol=22&iss=2&rank=13

Approximation by Superpositions of a Sigmoidal Function

We prove that approximative compactness of a Banach space X is equivalent to the conjunction of reflexivity and the Kadec-Klee property of X. This means that approximative compactness coincides with the drop property defined by Rolewicz in Studia Math. 85 (1987), 25 – 35. Using this general result we find criteria for approximative compactness in the class of Musielak–Orlicz function and sequence spaces for both (the Luxemburg norm and the Amemiya norm) as well as critria for this property in the class of Lorentz–Orlicz spaces. Criteria for full rotundity of Musielak–Orlicz spaces are also presented in the case of the Luxemburg norm. An example of a reflexive strictly convex Köthe function space which is not approximatively compact and some remark concerning the compact faces property for Musielak–Orlicz spaces are given.

https://www.ems-ph.org/journals/show_pdf.php?issn=0232-2064&vol=25&iss=2&rank=3

Approximative Compactness and Full Rotundity in Musielak-Orlicz spaces and Lorentz-Orlicz spaces

Symmetric spaces with maximal projection constants

Approximation of functions of finite variation by superpositions of a Sigmoidal function

Let V be an n-dimensional real Banach space and let λ(V ) denote its absolute projection constant. For any N ∈ N with N ≥ n define λn = sup{λ(V ) : dim(V ) = n, V ⊂ l ∞ }, λn = sup{λ(V ) : dim(V ) = n}. A well-known Grünbaum conjecture [Trans. Amer. Math. Soc. 95 (1960)] says that

/pdf/a-proof-of-the-grunbaum-conjecture-bgqrwjdz33.pdf

Grzegorz Lewicki

Papers

Approximation by Superpositions of a Sigmoidal Function

Approximative Compactness and Full Rotundity in Musielak-Orlicz spaces and Lorentz-Orlicz spaces

Symmetric spaces with maximal projection constants

Approximation of functions of finite variation by superpositions of a Sigmoidal function

A proof of the Grünbaum conjecture