This paper shows how stochastic approximation (SA) can be used to construct maximum likelihood estimates of system parameters. The procedure described here relies on a derivative approximation other than the usual finite-difference approximation associated with a Kiefer-Wolfowitz SA procedure. This alternative derivative approximation requires fewer, by a factor equal to the dimension of the parameter vector being estimated, computations than the standard finite-difference approximation. Numerical evidence presented in the paper indicates that this SA procedure is, relative to a Kiefer-Wolfowitz procedure, most efficient when considering large-scale systems.

A Stochastic Approximation Technique for Generating Maximum Likelihood Parameter Estimates

ldentification procedures in the time domain are studied for multivariable systems whieh yield a model with uncertainty bounds on its parameters. This parametrie uneertainty ean either be stochastic or deterministic depending on the assumptions which have been made with respect to the noise disturbing the system, i.e. statistica! properties or bounded error characteristics. The classical prediefion error identification approach is briefly reviewed descrihing first the SISO model struetures and the (pseudo-) canonical extensions to the multivariable case. These multivariable model structures, however, are not very suitable for process identification in practice, especially in an industrial environment because of the structure uneertainty. Therefore a minimal polynomial model structure is adopted which spows interesting similarities with the SISO case. The model estimation is solved by non-linear least-squares optimization minimizing the sum-squared prediction errors where the tirstand second-order derivatives ean be eomputed analytically in a very efficient way. In. the classical approach the assumption is required that the process is contained in the defined model set in order to provide stochastic uncertainty bounds. In practice, however, only a low order process approximation is obtained. For the situation that the proeess is not eontained in the model set, an expression for the parameter covariance matrix is presenled taking the underrnodelling explicitly into account, which can be used to eompute stochastie uneertainty bounds on the parameters.

Parametric uncertainty in system identification

Maximum likelihood estimation of a mean and a covariance matrix whose structure is constrained only to general positive semi-definiteness is treated in this paper. Necessary and sufficient conditions for the local optimality of mean and covariance matrix estimates are given. Observations are assumed to be independent. When the observations are also assumed to be identically distributed, the optimality conditions are used to obtain the mean and covariance matrix solutions in closed form. For the nonidentically distributed observation case, a general numerical technique which integrates scoring and Newton's iterations to solve the optimality condition equations is presented, and convergence performance is examined.

Maximum likelihood mean and covariance matrix estimation constrained to general positive semi-definiteness

Daniel C. ChinThe Johns Hopkins University AppliedPhysics Laboratory11100 Johns Hopkins RoadLaurel, Maryland 20723-6099E-mail: daniel.chin@jhuapl.eduAbstract. Extracting a multivariate nonlinear physical model from a setof satellite images is considered as a multivariate nonlinear regressionproblem. Multiple local solutions often prevent gradient type algorithmsfrom obtaining global optimal solutions. A method of solving this problemis presented based on the simultaneous perturbation stochastic approxi-mation (SPSA) algorithm. The method is applied to a problem of estimat-ing the distribution of energetic ion populations in the magnetospherefrom global images of the magnetosphere. The approach uses multipleobjective functions: single image errors and the summation of squareimage errors. The algorithm is demonstrated on simulated energetic-neutral atom (ENA) images. Within a reasonable number of functionevaluations, the process converges and reconstructs the images with amean square error less than or equal to 0.1% of the original image. Also,the SPSA method is compared with results obtained from simulated an-nealing (SAN) in a single objective function setting. In the comparisonstudy, SPSA has a 3:1 advantage over SAN in both accuracy and efﬁ-ciency measures.

Simultaneous perturbation method for processing magnetospheric images

This bibliography on the Kalman filter was obtained from the Cornell University Library Resources computer network. The four indexes used were Agricola, Biosis pre-1993, Biosis post-1992, and CAB. The references to Kalman filtering found in these indexes are listed together as one bibliography in Section 2 of the paper. Identification of content for each citation is listed after the reference. The Current Index in Statistics was used to obtain the bibliography given in Section 3 . By not combining the Current Index with the others, it is possible to observe the completeness of the Current Index relative to the others. References in the physical sciences fields for the most part are not included in this bibliography unless they appeared in these indexes. References on applications for the most part come from the ·fields of agriculture, biology, and medicine.

/pdf/kalman-filter-bibliography-agriculture-biology-and-medicine-2gcah6s8c6.pdf

Kalman Filter Bibliography: Agriculture, Biology, and Medicine

A method is presented for constrained maximum likelihood identification of the (positive semi-definite) initial covariance of an otherwise known linear discrete time dynamical system, with guaranteed positive semi-definite estimate at each step. The technique is a modification of Newton-Raphson or Scoring procedures transformed linearly to the space of the Cholesky square root matrix. The required algorithm is specified completely, and numerical and analytic difficulties and their solutions are discussed. It is shown that in cases of interest this procedure can result in order of magnitude reduction in computational costs compared to other iterative ML schemes which guarantee a semi-definite covariance estimate at each step. Formal extension to the maximum likelihood identification of time constants and power spectral densities is presented.

Efficient Maximum Likelihood Identification of a Positive Semi-Definite Covariance of Initial Population Statistics

A methodology is presented to account for the uncertainty in maximum likelihood estimates of state space parameters in the presence of uncertain nuisance parameters. The technique uses the asymptotic normality of the uncertainty in the estimates and the implicit function theorem to determine a correction to the estimate uncertainty evaluated from the Fisher information matrix. Efficient evaluation of the correction using Kalman filters is discussed and a numerical example for the X-22A aircraft is presented.

Identification of State-Space Parameters in the Presence of Uncertain Nuisance Parameters

In order to fulfll the mison objectives of a space platform in the evet dcomponnt filures, an integrated space platform controlrsteh is required. The research summaisd here deronstes the need for providing robust control sytem reconfiguration by showin.g how faiure of snsor and actuator components can degrade the control systm performance and lead in many ca to closed-loop instability without reconfiguration. The key elements of a space platform control reconfiguration system are identified and discue The concept is simulated on a simple structural model to the pinciples related to reconfiguration control system design and its impact on system performance. A fault detection logic based on likelfihood ratio tehniqusisp ted for detection and iaolation of nsor and actuator failures. State estimation using full state and reduced state obsevers is discussed and a full state observer i's irplemented in the simulations. On-board control sytem reconfiguration scheme are discussed for the implementationof Discrete Linear Quadratic Gssian (DLQG) control. The performance of the integrated reconfigu ration sytem is demonstrated to be superior to robust systems without reconfiguration capabilities.

John P. Garner

Papers

Efficient Maximum Likelihood Identification of a Positive Semi-Definite Covariance of Initial Population Statistics

Identification of State-Space Parameters in the Presence of Uncertain Nuisance Parameters

Robust Algorithms for Control and Survivability Enhancement of Space Platforms