# Showing papers in "IEEE Transactions on Automatic Control in 1965"

••

TL;DR: In this article, an adaptive approach to the problem of estimating a sampled, stochastic process described by an initially unknown parameter vector is presented, which is composed of a set of elemental estimators and a corresponding set of weighting coefficients, one pair for each possible value of the parameter vector.

Abstract: This work presents an adaptive approach to the problem of estimating a sampled, stochastic process described by an initially unknown parameter vector. Knowledge of this quantity completely specifies the statistics of the process, and consequently the optimal estimator must "learn" the value of the parameter vector. In order that construction of the optimal estimator be feasible it is necessary to consider only those processes whose parameter vector comes from a finite set of a priori known values. Fortunately, many practical problems may be represented or adequately approximated by such a model. The optimal estimator is found to be composed of a set of elemental estimators and a corresponding set of weighting coefficients, one pair for each possible value of the parameter vector. This structure is derived using properties of the conditional mean operator. For Gauss-Markov processes the elemental estimators are linear, dynamic systems, and evaluation of the weighting coefficients involves relatively simple, nonlinear calculations. The resulting system is optimum in the sense that it minimizes the expected value of a positive-definite, quadratic form in terms of the error (a generalized mean-square-error criterion). Because the system described in this work is optimal, it differs from previous attempts at adaptive estimation, all of which have used approximation techniques or sub-optimal, sequential, optimization procedures [12], [13], and [14].

787 citations

••

TL;DR: In this paper, an iterative technique is proposed to identify a linear system from samples of its input and output in the presence of noise by minimizing the mean-square error between system and model outputs.

Abstract: An iterative technique is proposed to identify a linear system from samples of its input and output in the presence of noise by minimizing the mean-square error between system and model outputs. The model chosen has a transfer function which is a ratio of polynomials in z-1. Although the regression equations for the optimal set of coefficients are highly nonlinear and intractable, it is shown that the problem can be reduced to the repeated solution of a related linear problem. Computer simulation of a number of typical discrete systems is used to demonstrate the considerable improvement over the Kalman estimate which can be obtained in a few iterations. The procedure is found to be effective at signal-to-noise ratios less than unity, and with as few as 200 samples of the input and output records.

660 citations

••

TL;DR: In this article, conditions for capture and for optimality are derived for a class of optimal pursuit-evasion problems, and results are used to demonstrate that the well-known proportional navigation law is actually an optimal intercept strategy.

Abstract: In this paper it is shown that variational techniques can be applied to solve differential games. Conditions for capture and for optimality are derived for a class of optimal pursuit-evasion problems. Results are used to demonstrate that the well-known proportional navigation law is actually an optimal intercept strategy.

495 citations

••

TL;DR: In this paper, it is shown that if correlation times are not short, or if some measurements are free of noise, the optimal filter is a modification of the Kalman-Bucy filter which, in general, contains differentiators as well as integrators.

Abstract: The Kalman-Bucy filter for continuous linear dynamic systems assumes all measurements contain "white" noise, i.e. noise with correlation times short compared to times of interest in the system. It is shown here that if correlation times are not short, or if some measurements are free of noise, the optimal filter is a modification of the Kalman-Bucy filter which, in general, contains differentiators as well as integrators. It is also shown for this case that the estimate and its covariance matrix are, in general, discontinuous at the time when measurements are begun. The case of random bias errors in the measurements is shown by example to be a limiting case of colored noise.

333 citations

••

TL;DR: In this article, a new expression for the transfer function of a system described by a set of first-order differential equations is given; this expression not only relates the poles and zeros to the eigenvalues of matrices but also makes it possible to compute the transfer functions without matrix inversion.

Abstract: This paper is concerned with the relationships between time and frequency domain descriptions of linear, time-invariant systems and with the evaluation of the effects of feedback on such systems A new expression for the transfer function of a system described by a set of first-order differential equations is given; this expression not only relates the poles and zeros to the eigenvalues of matrices but also makes it possible to compute the transfer function without matrix inversion The effects of state variable feedback on controllability, observability, and pole-zero configurations are discussed and the effects of feeding back the output and its derivatives are considered The application of these ideas to an optimal control problem is sketched and methods of extending them to the multi-input, multi-output case are examined

187 citations

••

TL;DR: A learning control system using a reinforcement technique that is capable of controlling a plant that may be nonlinear and nonstationary and which learns the best control choice for each control situation.

Abstract: This paper describes a learning control system using a reinforcement technique. The controller is capable of controlling a plant that may be nonlinear and nonstationary. The only a priori information required by the controller is the order of the plant. The approach is to design a controller which partitions the control measurement space into sets called control situations and then learns the best control choice for each control situation. The control measurements are those indicating the state of the plant and environment. The learning is accomplished by reinforcement of the probability of choosing a particular control choice for a given control situation. The system was stimulated on an IBM 1710-GEDA hybrid computer facility. Experimental results obtained from the simulation are presented.

149 citations

••

TL;DR: A method of analyzing large, multiple-loop control systems which can be represented by simultaneous, linear, first-order, differential equations using the QR transform and the sensitivities of these eigenvalues to the system's parameters are found from the normal and transposed eigenvectors.

Abstract: This paper describes a method of analyzing large, multiple-loop control systems which can be represented by simultaneous, linear, first-order, differential equations. The eigenvalues of the system are found using the QR transform. Next, the sensitivities of these eigenvalues to the system's parameters are found from the normal and transposed eigenvectors of the system. Results from a 51st-order system to which this method has been applied are given as an example.

126 citations

••

TL;DR: In this paper, the authors used multidimensional Z transforms and the discrete form of the Volterra series to analyze a large class of nonlinear sampled-data systems and nonlinear difference equations, presenting the solution in terms of the kernels of the VOLTERRA series.

Abstract: By utilizing multidimensional Z transforms and the discrete form of the Volterra series it is shown how to analyze a large class of nonlinear sampled-data systems and nonlinear difference equations, presenting the solution in terms of the kernels of the Volterra series. The method is shown to be just as applicable when the discrete system is not quiescent with the interaction between the initial conditions and the driving function evidenced by means of the transition matrix. Several illustrative examples are given, and applications of the method are suggested.

98 citations

••

TL;DR: In this article, a new class of pulse-frequency modulated systems is presented, referred to here as ''SigmaPFM'' which have many advantages over previously used schemes such as integral PFM.

Abstract: A new class of pulse-frequency modulated systems is presented in this paper. These systems referred to here as \SigmaPFM have many advantages over previously used schemes such as integral PFM. Most significant advantages are improved stability and simpler physical implementation of the modulator. The major part of this paper is concerned with the study of sustained oscillations using a specially developed quasi-describing function. One important feature of these kinds of PFM systems is that they often present a limit annulus and not a limit cycle, a feature which is common in most nonlinear discrete feedback systems. A few examples with experimental verification are presented and the limitations of the method are discussed.

92 citations

••

TL;DR: A global search of a multimodal noisy performance surface using a probabilistic automaton as a model that has a variable structure so that the system is able to adjust its search probabilities continuously.

Abstract: This paper discusses a global search of a multimodal noisy performance surface using a probabilistic automaton as a model. The various regions are searched in accordance with probabilities assigned on the basis of past relative performances. The automaton has a variable structure so that the system is able to adjust its search probabilities continuously, and linear reinforcement is used as an averaging technique. The chief advantages of this procedure are the variable search probabilities and the simplicity of implementing the search (minimum amount of computations). The procedure is extended to a multidimensional case and examples are shown.

••

TL;DR: In this article, the effect of errors in the a priori information is studied when the sequential estimations are carried out on the states of linear systems disturbed by white noise, and four theorems are derived to describe the mutual relations among the three covariance matrices, namely the optimum, calculated, and actual covariance matrix.

Abstract: In this paper, the effect of errors in the a priori information is studied when the sequential estimations are carried out on the states of linear systems disturbed by white noise. Four theorems are derived to describe the mutual relations among the three covariance matrices, namely the optimum, calculated, and actual covariance matrices, where the last two are based on the incorrect a priori information. By finding the upper bound for the variance of the actual estimate, performance of the Kalman filter is prescribed and the knowledge is utilized for design of the combined system of analog and digital filters. A phase-locked loop receiver is used as an example of analog filter and the considerable improvement on the estimation process is deduced by the theory and it is confirmed by the experimental simulation on the digital computer.

••

TL;DR: In this article, the authors illustrate the limitations of the generalized Popov Theorem in establishing the stability of a loop containing a single nonlinearity, and use the Liapunov Theory to give a new frequency domain stability criterion for such systems.

Abstract: The objective of this paper is to illustrate the limitations of the generalized Popov Theorem in establishing the stability of a loop containing a single nonlinearity, and to use the Liapunov Theory to give a new frequency domain stability criterion for such systems. The new criterion differs from Popov's in that less restrictive assumptions are made on the linear part, and stronger assumptions are made on the nonlinearity. In this paper, it is assumed that the nonlinearity is monotone increasing. The approach used here is quite general, however, and in a companion paper various other restrictions are considered.

••

TL;DR: In this paper, an automatic, finite-step numerical procedure is described for finding exact solutions on nonlinear optimal programming problems, which represents a unification and extension of the steepest-descent, and second variation techniques.

Abstract: : An automatic, finite-step numerical procedure is described for finding exact solutions on nonlinear optimal programming problems. The procedure represents a unification and extension of the steepest-descent, and second variation techniques. The procedure requires the backward integration of the usual adjoint-vector differential equations plus certain matrix differential equations. These integrations correspond, in the ordinary calculus, to finding the first and second derivatives of the performance index respectively. The matrix equations arise from an inhomogeneous Ricatti transformation, which generates a linear 'feedback control law' that preserves the gradient histories on the next step or permits changing them by controlled amounts, while also changing terminal conditions by controlled amounts. Thus, in a finite number of steps, the gradient histories can be made identically zero, as required for optimality, and the terminal conditions satisfied exactly. One forward plus one backward sweep, correspond to one step in the Newton-Raphson technique for finding maxima and minima in the ordinary calculus. As by-products, the procedure produces (a) the functions needed to show that the program is, or is not, a local maximum (the generalized Jacobi test) and (b) the feedback gain programs for neighboring optimal paths to the same, or a slightly different, set of terminal conditions.

••

••

TL;DR: In this article, a new definition of relative sensitivity is introduced for the optimal control problem, wherein the system performance is always compared with its optimum under the given circumstances, and the implications of the relative sensitivity and its relevance to optimal system design are discussed in detail.

Abstract: The sensitivity of a control system is usually taken to be the normalized variation of some desired characteristic with the variation of plant or controller parameters. Rather than the usual absolute sensitivity described above, a new definition of relative sensitivity is introduced for the optimal control problem, wherein the system performance is always compared with its optimum under the given circumstances. The implications of the relative sensitivity and its relevance to optimal system design are discussed in detail. Moreover, a theoretical approach to the problem of system optimization when plant parameters are subject to change or are incompletely specified is presented.

••

TL;DR: In this article, the tendency of human operators to respond in a bang-bang fashion when controlling high-order systems is investigated, and a simple on-off model for the human is suggested, and the switching lines and error trajectories in the phase plane are presented.

Abstract: The tendency of many human operators to respond in a bang-bang fashion when controlling some high-order systems is investigated. A three-mode switch is compared with a linear control stick and shown to permit better manual control of some systems with more lag than double integration. In experiments requiring stabilization of a moving base flight simulator programmed as an unstable system (undamped inverted pendulum), operators use the linear control stick in a bang-bang fashion. In place of quasi-linear models for these situations, a simple on-off model for the human is suggested, and the switching lines and error trajectories in the phase plane are presented. The ability to control an unstable system with visual and motion cues is compared.

••

TL;DR: In this paper, the sensitivity of the performance of optimal control systems to plant parameter variations is discussed, and the problem of determining the change in value or a performance index with parameter variations, and a method or computing the performance index sensitivity functions is considered.

Abstract: Publisher Summary This chapter discusses the sensitivity of the performance of optimal control systems to plant parameter variations. The problem or determining the change in value or a performance index with parameter variations, and outlined a method or computing the performance index sensitivity functions is considered. The system to be controlled is described by the vector differential equation x = f (t, x, u, a), x(t0) = x0, where x is a n-dimensional column vector representing the state of the plant and u is an r-dimensional column vector representing the control input. The vector a = (a1, a2, …, am) represents a set of plant parameters. The chapter also discusses closed- and open-loop systems.

••

••

TL;DR: The Modulating Functions Method (MFM) as mentioned in this paper is a method of extracting unknown coefficients by using definite integrals involving only recorded functions, and none of their derivatives, and it can be used for process identification.

Abstract: More results are given about a new method of Process Identification. This method, called "Modulating Functions Method" had been described in a previous paper by the same authors [5]. It essentially consists of extracting unknown coefficients by using definite integrals involving only recorded functions, and none of their derivatives. New formulas concern time-varying coefficients and pure delays.

••

TL;DR: Borders for single channel sampling are obtained for such cases where the jitter is independent from sample to sample and where theJitter is correlated, and an error bound for a special case of band-pass sampling using two samplers is derived.

Abstract: When timing jitter is present in a sampled-data system, it is important to be able to calculate the error introduced by the jittered sampler. However, a detailed statistical description of the jitter is not usually available to allow the determination of jitter error and the optimum filters. This paper calculates bounds for such error. For single channel sampling, bounds are obtained for such cases where the jitter is independent from sample to sample and where the jitter is correlated. An error bound for a special case of band-pass sampling using two samplers is also derived. In all cases, the signal is assumed to be properly bandlimited, so that, in the absence of jitter, the system is free of error. The final expressions of the error bounds depend, regardless of other jitter statistics, only on the type and the variance of the jitter.

••

••

••

••

TL;DR: In this article, a generalization of Hurwitz, Nyquist, and Mikhailov stability criteria for linear-feedback systems is presented, which greatly simplify the analysis procedure and make it convenient for computer applications.

Abstract: This paper presents a generalization of Hurwitz, Nyquist, and Mikhailov stability criteria for investigations of relative stability of linear-feedback systems. The generalization utilizes the Chebyshev functions, which greatly simplify the analysis procedure and make it convenient for computer applications. The use of the generalized stability criteria is illustrated by examples.

••