One of the core problems of modern statistics is to approximate difficult-to-compute probability densities. This problem is especially important in Bayesian statistics, which frames all inference about unknown quantities as a calculation involving the posterior density. In this article, we review variational inference (VI), a method from machine learning that approximates probability densities through optimization. VI has been used in many applications and tends to be faster than classical methods, such as Markov chain Monte Carlo sampling. The idea behind VI is to first posit a family of densities and then to find a member of that family which is close to the target density. Closeness is measured by Kullback–Leibler divergence. We review the ideas behind mean-field variational inference, discuss the special case of VI applied to exponential family models, present a full example with a Bayesian mixture of Gaussians, and derive a variant that uses stochastic optimization to scale up to massive data...

Variational Inference: A Review for Statisticians

The Theory of Matrices. By F R. Gantmacher. Two volumes, pp. 374 and 276. 1959. (Translated from the Russian by K. A. Hirsch; Chelsea Publishing Company, New York)

General Topology-I

A class of statistical models is proposed for longitudinal network data. The dependent variable is the changing (or evolving) relation network, represented by two or more observations of a directed graph with a fixed set of actors. The network evolution is modeled as the consequence of the actors making new choices, or withdrawing existing choices, on the basis of functions, with fixed and random components, that the actors try to maximize. Individual and dyadic exogenous variables can be used as covariates. The change in the network is modeled as the stochastic result of network effects (reciprocity, transitivity, etc.) and these covariates. The existing network structure is a dynamic constraint for the evolution of the structure itself. The models are continuous-time Markov chain models that can be implemented as simulation models. The model parameters are estimated from observed data. For estimating and testing these models, statistical procedures are proposed that are based on the method of moments. The statistical procedures are implemented using a stochastic approximation algorithm based on computer simulations of the network evolution process.

/pdf/the-statistical-evaluation-of-social-network-dynamics-4c1uefq1fj.pdf

The statistical evaluation of social network dynamics

The purpose of this survey is to review parameter identification procedures in groundwater hydrology and to examine computational techniques which have been developed to solve the inverse problem. Parameter identification methods are classified under the error criterion used in the formulation of the inverse problem. The problem of ill-posedness in connection with the inverse problem is addressed. Typical inverse solution techniques are highlighted. The review also includes the evaluation of methods used for computing the sensitivity matrix. Statistics which can be used to estimate the parameter uncertainty are outlined. Attempts have been made to compare and contrast representative inverse procedures, and direction for future research is suggested.

Review of Parameter Identification Procedures in Groundwater Hydrology: The Inverse Problem

This study presents the conditions of applicability of stochastic approximation algorithms that minimize a mean-square error criterion for identification of a linear discrete-time stationary system without dynamical numerator. The acceleration of the convergence is discussed. Then a tentative is outlined to overcome the previous requirement of states accessibility.

Stochastic approximation algorithms and applications

Sensors and controllers location in distributed systems-A survey

This report presents a survey of methods for the Distributed Parameter System Identification Problem. Fundamental concepts such as ‘ System Identification ’ and ‘ Distributed Parameter Systems ’ are briefly summarized in order to make precise what kind of problem will be considered here. The various identification methods are grouped into three disjoint classes, namely : ‘ Direct Method ’, ‘ Reduction to a Lumped Parameter System ’ and ‘ Reduction to an Algebraic Equation ’. Under this classification we give a general survey of the main approaches to the problem of identification in distributed systems

Distributed parameter system indentification A survey

Preface * Set Theoretic Structures * Algebraic Structures * Topological Structures * Banach Spaces * Hilbert Spaces * The Spectral Theorem * References * Index

Elements of Operator Theory

Decompositions and models for Hilbert-space operators have been active research topics in recent years, and this book is intended as an introduction to this area of operator theory, working from an abstract point of view. The approach is elementary in that all the proofs only use standard results of single operator thoery, although many of the questions posed in the text lead on to open problems.

Carlos S. Kubrusly

Papers

Stochastic approximation algorithms and applications

Sensors and controllers location in distributed systems-A survey

Distributed parameter system indentification A survey

Elements of Operator Theory

An introduction to models and decompositions in operator theory