Most of the signal processing that we will study in this course involves local operations on a signal, namely transforming the signal by applying linear combinations of values in the neighborhood of each sample point. You are familiar with such operations from Calculus, namely, taking derivatives and you are also familiar with this from optics namely blurring a signal. We will be looking at sampled signals only. Let's start with a few basic examples. Local difference Suppose we have a 1D image and we take the local difference of intensities, DI(x) = 1 2 (I(x + 1) − I(x − 1)) which give a discrete approximation to a partial derivative. (We compute this for each x in the image.) What is the effect of such a transformation? One key idea is that such a derivative would be useful for marking positions where the intensity changes. Such a change is called an edge. It is important to detect edges in images because they often mark locations at which object properties change. These can include changes in illumination along a surface due to a shadow boundary, or a material (pigment) change, or a change in depth as when one object ends and another begins. The computational problem of finding intensity edges in images is called edge detection. We could look for positions at which DI(x) has a large negative or positive value. Large positive values indicate an edge that goes from low to high intensity, and large negative values indicate an edge that goes from high to low intensity. Example Suppose the image consists of a single (slightly sloped) edge:

http://ieeexplore.ieee.org/iel5/5/31307/01456462.pdf

Linear systems

System Identification I

A dimension reduction method called discrete empirical interpolation is proposed and shown to dramatically reduce the computational complexity of the popular proper orthogonal decomposition (POD) method for constructing reduced-order models for time dependent and/or parametrized nonlinear partial differential equations (PDEs). In the presence of a general nonlinearity, the standard POD-Galerkin technique reduces dimension in the sense that far fewer variables are present, but the complexity of evaluating the nonlinear term remains that of the original problem. The original empirical interpolation method (EIM) is a modification of POD that reduces the complexity of evaluating the nonlinear term of the reduced model to a cost proportional to the number of reduced variables obtained by POD. We propose a discrete empirical interpolation method (DEIM), a variant that is suitable for reducing the dimension of systems of ordinary differential equations (ODEs) of a certain type. As presented here, it is applicable to ODEs arising from finite difference discretization of time dependent PDEs and/or parametrically dependent steady state problems. However, the approach extends to arbitrary systems of nonlinear ODEs with minor modification. Our contribution is a greatly simplified description of the EIM in a finite-dimensional setting that possesses an error bound on the quality of approximation. An application of DEIM to a finite difference discretization of the one-dimensional FitzHugh-Nagumo equations is shown to reduce the dimension from 1024 to order 5 variables with negligible error over a long-time integration that fully captures nonlinear limit cycle behavior. We also demonstrate applicability in higher spatial dimensions with similar state space dimension reduction and accuracy results.

/pdf/nonlinear-model-reduction-via-discrete-empirical-f3tjqvqhuf.pdf

Nonlinear Model Reduction via Discrete Empirical Interpolation

The subject of inverse problems in differential equations is of enormous practical importance, and has also generated substantial mathematical and computational innovation. Typically some form of regularization is required to ameliorate ill-posed behaviour. In this article we review the Bayesian approach to regularization, developing a function space viewpoint on the subject. This approach allows for a full characterization of all possible solutions, and their relative probabilities, whilst simultaneously forcing significant modelling issues to be addressed in a clear and precise fashion. Although expensive to implement, this approach is starting to lie within the range of the available computational resources in many application areas. It also allows for the quantification of uncertainty and risk, something which is increasingly demanded by these applications. Furthermore, the approach is conceptually important for the understanding of simpler, computationally expedient approaches to inverse problems.

/pdf/inverse-problems-a-bayesian-perspective-1dknxttxjb.pdf

Inverse problems: A Bayesian perspective

Numerical simulation of large-scale dynamical systems plays a fundamental role in studying a wide range of complex physical phenomena; however, the inherent large-scale nature of the models often leads to unmanageable demands on computational resources. Model reduction aims to reduce this computational burden by generating reduced models that are faster and cheaper to simulate, yet accurately represent the original large-scale system behavior. Model reduction of linear, nonparametric dynamical systems has reached a considerable level of maturity, as reflected by several survey papers and books. However, parametric model reduction has emerged only more recently as an important and vibrant research area, with several recent advances making a survey paper timely. Thus, this paper aims to provide a resource that draws together recent contributions in different communities to survey the state of the art in parametric model reduction methods. Parametric model reduction targets the broad class of problems for wh...

/pdf/a-survey-of-projection-based-model-reduction-methods-for-3yjqbxk305.pdf

A Survey of Projection-Based Model Reduction Methods for Parametric Dynamical Systems

Balanced truncation is one of the most common model reduction schemes. In this note, we present a survey of balancing related model reduction methods and their corresponding error norms, and also introduce some new results. Five balancing methods are studied: (1) Lyapunov balancing, (2) stochastic balancing, (3) bounded real balancing, (4) positive real balancing and (5) frequency weighted balancing. For positive real balancing, we introduce a multiplicative-type error bound. Moreover, for a certain subclass of positive real systems, a modified positive-real balancing scheme with an absolute error bound is proposed. We also develop a new frequency-weighted balanced reduction method with a simple bound on the error system based on the frequency domain representations of the system gramians. Two numerical examples are illustrated to verify the efficiency of the proposed methods.

A survey of model reduction by balanced truncation and some new results

An overview of model reduction methods and a comparison of the resulting algorithms is presented These approaches are divided into two broad categories, namely SVD based and moment matching based methods It turns out that the approximation error in the former case behaves better globally in frequency while in the latter case the local behavior is better

/pdf/a-survey-of-model-reduction-methods-for-large-scale-systems-1ewzejlosw.pdf

A Survey of Model Reduction Methods for Large-Scale Systems

The optimal $\mathcal{H}_2$ model reduction problem is of great importance in the area of dynamical systems and simulation. In the literature, two independent frameworks have evolved focusing either on solution of Lyapunov equations on the one hand or interpolation of transfer functions on the other, without any apparent connection between the two approaches. In this paper, we develop a new unifying framework for the optimal $\mathcal{H}_2$ approximation problem using best approximation properties in the underlying Hilbert space. This new framework leads to a new set of local optimality conditions taking the form of a structured orthogonality condition. We show that the existing Lyapunov- and interpolation-based conditions are each equivalent to our conditions and so are equivalent to each other. Also, we provide a new elementary proof of the interpolation-based condition that clarifies the importance of the mirror images of the reduced system poles. Based on the interpolation framework, we describe an iteratively corrected rational Krylov algorithm for $\mathcal{H}_2$ model reduction. The formulation is based on finding a reduced order model that satisfies interpolation-based first-order necessary conditions for $\cHtwo$ optimality and results in a method that is numerically effective and suited for large-scale problems. We illustrate the performance of the method with a variety of numerical experiments and comparisons with existing methods.

/pdf/mathcal-h-2-model-reduction-for-large-scale-linear-dynamical-xoow2ovn5b.pdf

$\mathcal{H}_2$ Model Reduction for Large-Scale Linear Dynamical Systems

United States. Air Force Office of Scientific Research (Computational Mathematics Grant FA9550-12-1-0420)

Serkan Gugercin

Papers

A Survey of Projection-Based Model Reduction Methods for Parametric Dynamical Systems

A survey of model reduction by balanced truncation and some new results

A Survey of Model Reduction Methods for Large-Scale Systems

$\mathcal{H}_2$ Model Reduction for Large-Scale Linear Dynamical Systems

A Survey of Projection-Based Model Reduction Methods for Parametric Dynamical Systems