Showing papers by "Amir Dembo published in 1991"

Information theoretic inequalities

Abstract: The role of inequalities in information theory is reviewed, and the relationship of these inequalities to inequalities in other branches of mathematics is developed. The simple inequalities for differential entropy are applied to the standard multivariate normal to furnish new and simpler proofs of the major determinant inequalities in classical mathematics. The authors discuss differential entropy inequalities for random subsets of samples. These inequalities when specialized to multivariate normal variables provide the determinant inequalities that are presented. The authors focus on the entropy power inequality (including the related Brunn-Minkowski, Young's, and Fisher information inequalities) and address various uncertainty principles and their interrelations. >

High-order absolutely stable neural networks

Abstract: The stability properties of arbitrary order continuous-time dynamic neural networks are studied in the spirit of an earlier analysis of a first-order system by M.A. Cohen and S. Grossberg (1983). The corresponding class of Lyapunov function is presented and the equilibrium points are characterized. The relationships with other continuous-time models are pointed out. >

Strong Limit Theorems of Empirical Functionals for Large Exceedances of Partial Sums of I.I.D. Variables

Abstract: Let $(X_i,U_i)$ be pairs of iid bounded real-valued random variables ($X_i$ and $U_i$ are generally mutually dependent) Assume $E\lbrack X_i\rbrack 0\} > 0$ For the (rare) partial sum segments where $\sum^l_{i=k}X_i \rightarrow \infty$, strong limit laws are derived for the sums $\sum^l_{i=k}U_i$ In particular a strong law for the length $(l - k + 1)$ and the empirical distribution of $U_i$ in the event of large segmental sums of $\sum X_i$ are obtained Applications are given in characterizing the composition of high scoring segments in letter sequences and for evaluating statistical hypotheses of sudden change points in engineering systems

Strong Limit Theorems of Empirical Distributions for Large Segmental Exceedances of Partial Sums of Markov Variables

Abstract: Let $A_1,A_2,\ldots,A_n$ be generated governed by an $r$-state irreducible Markov chain and suppose $(X_i,U_i)$ are real valued independently distributed given the sequence $A_1,A_2,\ldots,A_n$, where the joint distribution of $(X_i,U_i)$ depends only on the values of $A_{i-1}$ and $A_i$ and is of bounded support Where $A_0$ is started with its stationary distribution, $E\lbrack X_1\rbrack 0, m = 1,\ldots,k; C\} > 0$ is assumed For the partial sum realizations where $\sum^l_{i=k}X_i \rightarrow \infty$, strong laws are derived for the sums $\sum^l_{i=k}U_i$ Examples with $r = 2, X \in \{-1, 1\}$ and the cases of Brownian motion and Poisson process with negative drift are worked out

A general weight matrix formulation using optimal control

Abstract: Classical methods from optimal control theory are used in deriving general forms for neural network weights. The network learning or application task is encoded in a performance index of a general structure. Consequently, different instances of this performance index lead to special cases of weight rules, including some well-known forms. Comparisons are made with the outer product rule, spectral methods, and recurrent back-propagation. Simulation results and comparisons are presented. >