Journal ArticleDOI
Stochastic Approximation Algorithms for System Identification, Estimation, and Decomposition of Mixtures
Reads0
Chats0
TLDR
It is shown that the least-square-error fit of the measured output signals of the systems offers a recursive formula which is a special case of the proposed algorithm, and the rate of convergence is computed.Abstract:
A stochastic approximation procedure that minimizes a mean-square-error criterion is proposed in this paper It is applied first to derive an algorithm for recursive estimation of the mean-square-error approximation of the function which relates the input signals and the responses of a memoryless system The input signals are assumed to be generated at random with an unknown probability density function, and the response is measured with an error which has zero mean and finite variance A performance index for evaluating the rate of convergence of the algorithm is defined and then the optimal form of the algorithm is derived It is shown that the least-square-error fit of the measured output signals of the systems offers a recursive formula which is a special case of the proposed algorithm A recursive formula for estimation of a priori probabilities of the pattern classes using unclassified samples is then presented The rate of convergence is computed A minimum square-error estimate of a continuous probability density function is also obtained by the same algorithmread more
Citations
More filters
Journal ArticleDOI
An algorithm for linearly constrained adaptive array processing
TL;DR: A constrained least mean-squares algorithm has been derived which is capable of adjusting an array of sensors in real time to respond to a signal coming from a desired direction while discriminating against noises coming from other directions.
Journal ArticleDOI
Analysis of recursive stochastic algorithms
TL;DR: It is shown how a deterministic differential equation can be associated with the algorithm and examples of applications of the results to problems in identification and adaptive control.
Journal ArticleDOI
Learning control systems--Review and outlook
TL;DR: The basic concept of learning control is introduced, and the following five learning schemes are briefly reviewed: 1) trainable controllers using pattern classifiers, 2) reinforcement learning control systems, 3) Bayesian estimation, 4) stochastic approximation, and 5) Stochastic automata models.
Journal ArticleDOI
Learning Control Systems-Review and Outlook
TL;DR: The basic concept of learning control is introduced, and the following five learning schemes are briefly reviewed: 1) trainable controllers using pattern classifiers, 2) reinforcement learning control systems, 3) Bayesian estimation, 4) stochastic approximation, and 5) Stochastic automata models.
Journal ArticleDOI
Analysis of a general recursive prediction error identification algorithm
TL;DR: It is proved that this class of methods has the same convergence properties as its off-line counterparts under mild and general assumptions, and may also serve as a basis for unified description of many recursive identification methods.
References
More filters
On Stochastic Approximation
TL;DR: In this article, it was shown that the mean-square convergence of the Kiefer-Wolfowitz and Robbins-Monro schemes is in fact a special case of the self-correcting deterministic method.
Journal ArticleDOI
Learning without a teacher
TL;DR: This is a review paper discussing and comparing most of the approaches to learning without a teacher which have been suggested to date, and divided into six classes: guessing a sequence of hypotheses, modifications of this first approach, approximating probability densities by others more easily computed, estimating parameters of a known decision rule, theoretically exact methods which require approximations in implementation, and some miscellaneous approaches.
Journal ArticleDOI
Learning to recognize patterns in a random environment
N. Abramson,D. Braverman +1 more
TL;DR: It is shown that the use of prior observations for pattern recognition may be described as a process of learning the statistical characteristics of the patterns involved, and optimal systems are shown to consist of banks of generalized correlators.
Journal ArticleDOI
Recovery of functions from noisy measurements taken at randomly selected points and its application to pattern classification
R.L. Kashyap,C.C. Blaydon +1 more
TL;DR: Stochastic approximation theory to obtain algorithms for recovery of functions, using noisy measurements taken at randomly selected points, by exploiting the randomness of measurements.