Probability Distributions.- Linear Models for Regression.- Linear Models for Classification.- Neural Networks.- Kernel Methods.- Sparse Kernel Machines.- Graphical Models.- Mixture Models and EM.- Approximate Inference.- Sampling Methods.- Continuous Latent Variables.- Sequential Data.- Combining Models.

Pattern Recognition and Machine Learning

In this paper we develop new Newton and conjugate gradient algorithms on the Grassmann and Stiefel manifolds. These manifolds represent the constraints that arise in such areas as the symmetric eigenvalue problem, nonlinear eigenvalue problems, electronic structures computations, and signal processing. In addition to the new algorithms, we show how the geometrical framework gives penetrating new insights allowing us to create, understand, and compare algorithms. The theory proposed here provides a taxonomy for numerical linear algebra algorithms that provide a top level mathematical view of previously unrelated algorithms. It is our hope that developers of new algorithms and perturbation theories will benefit from the theory, methods, and examples in this paper.

http://www-math.mit.edu/~edelman/publications/geometry_of_algorithms.pdf

The Geometry of Algorithms with Orthogonality Constraints

The Theory of Matrices. By F R. Gantmacher. Two volumes, pp. 374 and 276. 1959. (Translated from the Russian by K. A. Hirsch; Chelsea Publishing Company, New York)

http://galton.uchicago.edu/~lekheng/courses/310w11/boyd-socp.pdf

Applications of second-order cone programming

Most of the signal processing that we will study in this course involves local operations on a signal, namely transforming the signal by applying linear combinations of values in the neighborhood of each sample point. You are familiar with such operations from Calculus, namely, taking derivatives and you are also familiar with this from optics namely blurring a signal. We will be looking at sampled signals only. Let's start with a few basic examples. Local difference Suppose we have a 1D image and we take the local difference of intensities, DI(x) = 1 2 (I(x + 1) − I(x − 1)) which give a discrete approximation to a partial derivative. (We compute this for each x in the image.) What is the effect of such a transformation? One key idea is that such a derivative would be useful for marking positions where the intensity changes. Such a change is called an edge. It is important to detect edges in images because they often mark locations at which object properties change. These can include changes in illumination along a surface due to a shadow boundary, or a material (pigment) change, or a change in depth as when one object ends and another begins. The computational problem of finding intensity edges in images is called edge detection. We could look for positions at which DI(x) has a large negative or positive value. Large positive values indicate an edge that goes from low to high intensity, and large negative values indicate an edge that goes from high to low intensity. Example Suppose the image consists of a single (slightly sloped) edge:

http://ieeexplore.ieee.org/iel5/5/31307/01456462.pdf

Linear systems

1. A Review of Some Required Concepts from Core Linear Algebra. 2. Floating Point Numbers and Errors in Computations. 3. Stability of Algorithms and Conditioning of Problems. 4. Numerically Effective Algorithms and Mathematical Software. 5. Some Useful Transformations in Numerical Linear Algebra and Their Applications. 6. Numerical Solutions of Linear Systems. 7. Least Squares Solutions to Linear Systems. 8. Numerical Matrix Eigenvalue Problems. 9. The Generalized Eigenvalue Problem. 10. The Singular Value Decomposition (SVD). 11. A Taste of Round-Off Error Analysis. Appendix A: A Brief Introduction to MATLAB. Appendix B: MATLAB and Selected MATLAB Programs.

Numerical Linear Algebra and Applications

The design and analysis of linear control systems: 

$$\dot x(t) = Ax(t) + Bu(t),\,\,\,\,\,\,\,\,\,\,y = Cx(t)$$

(1.1)

and 

$$x_{k + 1} = Ax_k + Bu_k ,\,\,\,\,\,yk = Cx_k$$

(1.2)

give rise to many interesting linear algebra problems. Some of the important ones are: controllability and observability problems, the problem of computing the exponential matrix e At ,the matrix equations problems: (Lyapunov equations, Sylvester equations, the algebraic Riccati equations), the pole-placement problems, stability problems, and frequency response problems. These problems have been very widely studied in the literature. There exists a voluminous work both on theory and computations. Theory is very rich. Unfortunately, the same can not be remarked about computations. Many of the earlier methods were developed before the computer era, and are not based on numerically sound techniques. Fortunately, in the last twenty years or so, numerically effective techniques have been developed for most of these problems, and numerical analysis aspects of these methods (e.g. study of stability by round-off error analysis) and of the problems themselves (e.g. study of sensitivity) have been studied.

Numerical methods for linear control systems

Introduction and Overview Review of Basic Concepts and Results from Theoretical Linear Algebra Fundamental Tools and Concepts from Numerical Linear Algebra Canonical Forms Obtained via Orthogonal Transformations Linear State Space Models and Solutions of the State Equations Contollability, Observability and Distance to Uncontrollability Stability, Inertia and Robust Stability Numerical Solutions and Conditioning of Lyapunov and Sylvester Equations Realization and Subspace Identification Feedback Stabilization, Eigenvalue Assignment and Optimal Control Numerical Methods and Conditioning of the Eigenvalue Assignment Problems State Estimation Numerical Solutions and Conditioning of Algebraic Riccati Equations Internal Balancing and Model Reduction Large-Scale Matrix Computations in Control: Krylov Subspace Methods Numerical Methods for Matrix-Second-Order Control Systems Existing Software for Control Systems Design and Analysis

Numerical methods for linear control systems : design and analysis

Orthogonality and partial pole assignment for the symmetric definite quadratic pencil

We propose two new methods for solution of the eigenvalue assignment problem associated with the second-order control system \global\hsize=30pc Specifically, the methods construct feedback matrices F 1 and F 2 such that the closed-loop quadratic pencil has a desired set of eigenvalues and the associated eigenvectors are well conditioned. Method 1 is a modification of the singular value decomposition-based method proposed by Juang and Maghami which is a second-order adaptation of the well-known robust eigenvalue assignment method by Kautsky et al. for first-order systems. Method 2 is an extension of the recent non-modal approach of Datta and Rincon for feedback stabilization of second-order systems. Robustness to numerical round-off errors is achieved by minimizing the condition numbers of the eigenvectors of the closed-loop second-order pencil. Control robustness to large plant uncertainty will not be explicitly considered in this paper. Numerical results for both the two methods are favourable. A compara...

Biswa Nath Datta

Papers

Numerical Linear Algebra and Applications

Numerical methods for linear control systems

Numerical methods for linear control systems : design and analysis

Orthogonality and partial pole assignment for the symmetric definite quadratic pencil

Numerically robust pole assignment for second-order systems