Showing papers by "Amir Dembo published in 1987"

A relaxation model for memory with high storage density.

Abstract: We present a relaxation model for memory based on a generalized coulomb potential. The model has arbitrarily large storage capacity and, in addition, well-defined basins of attraction about stored memory states. The model is compared with the Hopfield relaxation model.

A minimum discrimination information approach for hidden Markov modeling

Abstract: A new iterative approach for hidden Markov modeling of information sources which aims at minimizing the discrimination information (or the cross-entropy) between the source and the model is proposed. This approach does not require the commonly used assumption that the source to be modeled is a hidden Markov process. The algorithm is started from the model estimated by the traditional maximum likelihood (ML) approach and alternatively decreases the discrimination information over all probability distributions of the source which agree with the given measurements and all hidden Markov models. The proposed procedure generalizes the Baum algorithm for ML hidden Markov modeling. The procedure is shown to be a descent algorithm for the discrimination information measure and its local convergence is proved.

On the parameters estimation of continuous-time ARMA processes from noisy observations

Abstract: Recently, an iterative algorithm has been presented for estimating the parameters of partially observed continuous-time processes [1]. In this note we concentrate on continuous-time ARMA processes observed in white noise. A maximum a-posteriori (MAP) estimator is defined for the trajectory of the parameters' random process. This approach enables the MAP estimation of randomly slowly varying parameters, and extends the conventional treatment of time-invariant parameters. The iterative algorithm derived for the MAP estimation, increases the posterior probability of the parameters in each iteration, and converges to a stationary point of the posterior probability functional. Each iteration involves a standard linear smoother followed by a finite-dimensional linear system, and thus is easily implemented.

The design of optimal uniform filter banks with specified composite response

Abstract: In this paper we prove that for suitable design specifications and a wide class of optimization criteria, the optimal complex filter bank with specifications on its composite response is composed of frequency translated versions of a prototype filter. In particular, this holds for the min-max and WMMSE (weighted minimum mean square error) criteria. As a result, a simplified design problem whose solution is an optimal prototype filter is formulated. This prototype is essentially an optimal FIR low-pass filter subject to linear constraints on its impulse response. For the WMMSE criterion, this characterization of the optimal filter bank results in a simplified version of the design method presented in [1]. For the min-max criterion, this characterization implies that there exists an optimal window, by which the window design method results in the optimal low-pass prototype. The optimal window design problem is formulated as a linear programming problem, and an approximate solution is derived using the Remez exchange algorithm. For real filter banks in which each filter is composed of a pair of complex filters, the optimal filter bank is no longer composed of frequency translated versions of prototype filter. However, for efficient implementation, the prototype translation property may be part of the design specifications. For this reason, the optimal WMMSE prototype for a class of real filter banks is derived as well.

High density associative memories

Abstract: A class of high density associative memories is constructed, starting from a description of desired properties those should exhibit. These properties include high capacity, controllable basins of attraction and fast speed of convergence. Fortunately enough, the resulting memory is implementable by an artificial Neural Net.