The Kalman Filter (KF) is one of the most widely used methods for tracking and estimation due to its simplicity, optimality, tractability and robustness. However, the application of the KF to nonlinear systems can be difficult. The most common approach is to use the Extended Kalman Filter (EKF) which simply linearizes all nonlinear models so that the traditional linear Kalman filter can be applied. Although the EKF (in its many forms) is a widely used filtering strategy, over thirty years of experience with it has led to a general consensus within the tracking and control community that it is difficult to implement, difficult to tune, and only reliable for systems which are almost linear on the time scale of the update intervals. In this paper a new linear estimator is developed and demonstrated. Using the principle that a set of discretely sampled points can be used to parameterize mean and covariance, the estimator yields performance equivalent to the KF for linear systems yet generalizes elegantly to nonlinear systems without the linearization steps required by the EKF. We show analytically that the expected performance of the new approach is superior to that of the EKF and, in fact, is directly comparable to that of the second order Gauss filter. The method is not restricted to assuming that the distributions of noise sources are Gaussian. We argue that the ease of implementation and more accurate estimation features of the new filter recommend its use over the EKF in virtually all applications.

/pdf/new-extension-of-the-kalman-filter-to-nonlinear-systems-5fwidgi8qh.pdf

New extension of the Kalman filter to nonlinear systems

This paper describes a new approach for generalizing the Kalman filter to nonlinear systems. A set of samples are used to parametrize the mean and covariance of a (not necessarily Gaussian) probability distribution. The method yields a filter that is more accurate than an extended Kalman filter (EKF) and easier to implement than an EKF or a Gauss second-order filter. Its effectiveness is demonstrated using an example.

/pdf/a-new-method-for-the-nonlinear-transformation-of-means-and-17a2m82o66.pdf

A new method for the nonlinear transformation of means and covariances in filters and estimators

Recommender systems have become valuable resources for users seeking intelligent ways to search through the enormous volume of information available to them. One crucial unsolved problem for recommender systems is how best to learn about a new user. In this paper we study six techniques that collaborative filtering recommender systems can use to learn about new users. These techniques select a sequence of items for the collaborative filtering system to present to each new user for rating. The techniques include the use of information theory to select the items that will give the most value to the recommender system, aggregate statistics to select the items the user is most likely to have an opinion about, balanced techniques that seek to maximize the expected number of bits learned per presented item, and personalized techniques that predict which items a user will have an opinion about. We study the techniques thru offline experiments with a large pre-existing user data set, and thru a live experiment with over 300 users. We show that the choice of learning technique significantly affects the user experience, in both the user effort and the accuracy of the resulting predictions.

/pdf/getting-to-know-you-learning-new-user-preferences-in-332ov1yv44.pdf

Getting to know you: learning new user preferences in recommender systems

https://www.jstage.jst.go.jp/article/jmsj1965/75/1B/75_1B_257/_pdf

An Introduction to Estimation Theory (gtSpecial IssueltData Assimilation in Meteology and Oceanography: Theory and Practice)

Previous results on estimating errors or error bounds on identified transfer functions have relied on prior assumptions about the noise and the unmodeled dynamics. This prior information took the form of parameterized bounding functions or parameterized probability density functions, in the time or frequency domain with known parameters. It is shown that the parameters that quantify this prior information can themselves be estimated from the data using a maximum likelihood technique. This significantly reduces the prior information required to estimate transfer function error bounds. The authors illustrate the usefulness of the method with a number of simulation examples. How the obtained error bounds can be used for intelligent model-order selection that takes into account both measurement noise and under-modeling is shown. Another simulation study compares the method to Akaike's well-known FPE and AIC criteria. >

Quantifying the error in estimated transfer functions with application to model order selection

1 Basic Probability.- 1.1. Definitions.- 1.2. Probability Distributions and Densities.- 1.3. Expected Value, Covariance.- 1.4. Independence.- 1.5. The Radon-Nikodym Theorem.- 1.6. Continuously Distributed Random Vectors.- 1.7. The Matrix Inversion Lemma.- 1.8. The Multivariate Normal Distribution.- 1.9. Conditional Expectation.- 1.10. Exercises.- 2 Minimum Variance Estimation-How the Theory Fits.- 2.1. Theory Versus Practice-Some General Observations.- 2.2. The Genesis of Minimum Variance Estimation.- 2.3. The Minimum Variance Estimation Problem.- 2.4. Calculating the Minimum Variance Estimator.- 2.5. Exercises.- 3 The Maximum Entropy Principle.- 3.1. Introduction.- 3.2. The Notion of Entropy.- 3.3. The Maximum Entropy Principle.- 3.4. The Prior Covariance Problem.- 3.5. Minimum Variance Estimation with Prior Covariance.- 3.6. Some Criticisms and Conclusions.- 3.7. Exercises.- 4 Adjoints, Projections, Pseudoinverses.- 4.1. Adjoints.- 4.2. Projections.- 4.3. Pseudoinverses.- 4.4. Calculating the Pseudoinverse in Finite Dimensions.- 4.5. The Grammian.- 4.6. Exercises.- 5 Linear Minimum Variance Estimation.- 5.1. Reformulation.- 5.2. Linear Minimum Variance Estimation.- 5.3. Unbiased Estimators, Affine Estimators.- 5.4. Exercises.- 6 Recursive Linear Estimation (Bayesian Estimation).- 6.1. Introduction.- 6.2. The Recursive Linear Estimator.- 6.3. Exercises.- 7 The Discrete Kalman Filter.- 7.1. Discrete Linear Dynamical Systems.- 7.2. The Kalman Filter.- 7.3. Initialization, Fisher Estimation.- 7.4. Fisher Estimation with Singular Measurement Noise.- 7.5. Exercises.- 8 The Linear Quadratic Tracking Problem.- 8.1. Control of Deterministic Systems.- 8.2. Stochastic Control with Perfect Observations.- 8.3. Stochastic Control with Imperfect Measurement.- 8.4. Exercises.- 9 Fixed Interval Smoothing.- 9.1. Introduction.- 9.2. The Rauch, Tung, Streibel Smoother.- 9.3. The Two-Filter Form of the Smoother.- 9.4. Exercises.- Appendix A Construction Measures.- Appendix B Two Examples from Measure Theory.- Appendix C Measurable Functions.- Appendix D Integration.- Appendix E Introduction to Hilbert Space.- Appendix F The Uniform Boundedness Principle and Invertibility of Operators.

Estimation, Control, and the Discrete Kalman Filter

If one supposes a quantum logicL to be a σ-orthocomplete, orthomodular partially ordered set admitting a set of σ-orthoadditive functions (called states) fromL to the unit intervals [0, 1] such that these states distinguish the ordering and orthocomplement onL, then the observables onL are identified withL-valued measures defined on the Borel subsets of the real line. In this structure (and without the aid of Hilbert space formalism) the author shows that (1) the spectrum of an observable can be completely characterised by studying the observable (A−λ)−1, and (2) corresponding to every observableA there is a spectral resolution uniquely determined byA and uniquely determiningA.

Spectral theory in quantum logics

In Section 1.1, we alluded to the principle that titles this chapter. The idea of this principle, as we said there, is to assign probabilities in such a way that the resulting distribution contains no more information than is inherent in the data. The first attempt to do this was by Laplace and was called the “Principle of Insufficient Reason.” This principle said that two events should be assigned the same probability if there is no reason to do otherwise. This appears reasonable and is fine as far as it goes. However, Jaynes [12] said, “except in cases where there is an evident element of symmetry that clearly renders the events ‘equally possible,’ this assumption may appear just as arbitrary as any other that might be made.” What is needed, of course, is a way to uniquely quantify the “amount of uncertainty” represented by a probability distribution. As mentioned in Section 1.1, this was done in 1948 by C. E. Shannon in a paper entitled A Mathematical Theory of Communication [25]. Shannon was specifically interested in the problem of sen ding data through a noisy, discrete transmission channel. However, his results, most notably a rigorous treatment of entropy, have had far-reaching consequences in the theory of probability and statistics. We are not going to even attempt to summarize the broad effects his work has had nor reference the many papers that have since been written on the subject; our purpose is to quickly develop the notion of entropy and get on with the estimation problem of the last chapter. For more information on the subject of entropy, the reader is referred to the text by Ellis [4].

The Maximum Entropy Principle

In Chapter 6, we discussed the problem of making recursive estimates of a random vector X. The problem was static in the sense that every measurement was used to update or improve the estimate of the same random vector X. We now consider the case where the random vector changes in time, between measurements, according to a specified statistical dynamic.

The Discrete Kalman Filter

We now come to the final topic in this text, fixed interval smoothing. Just as we have seen with other topics in past chapters, this is an easy problem to state and very complex problem to solve.

Donald E. Catlin

Papers

Estimation, Control, and the Discrete Kalman Filter

Spectral theory in quantum logics

The Maximum Entropy Principle

The Discrete Kalman Filter

Fixed Interval Smoothing