Data Mining Practical Machine Learning Tools and Techniques

An overview of the self-organizing map algorithm, on which the papers in this issue are based, is presented in this article.

The Self-Organizing Map

1. Basic Concepts. 2. Nonparametric Methods. 3. Parametric Methods for Rational Spectra. 4. Parametric Methods for Line Spectra. 5. Filter Bank Methods. 6. Spatial Methods. Appendix A: Linear Algebra and Matrix Analysis Tools. Appendix B: Cramer-Rao Bound Tools. Appendix C: Model Order Selection Tools. Appendix D: Answers to Selected Exercises. Bibliography. References Grouped by Subject. Subject Index.

/pdf/spectral-analysis-of-signals-4dbupf9m4n.pdf

Spectral analysis of signals

The performance of the MUSIC and ML methods is studied, and their statistical efficiency is analyzed. The Cramer-Rao bound (CRB) for the estimation problems is derived, and some useful properties of the CRB covariance matrix are established. The relationship between the MUSIC and ML estimators is investigated as well. A numerical study is reported of the statistical efficiency of the MUSIC estimator for the problem of finding the directions of two plane waves using a uniform linear array. An exact description of the results is included. >

MUSIC, maximum likelihood, and Cramer-Rao bound

We explore in this chapter questions of existence and uniqueness for solutions to stochastic differential equations and offer a study of their properties. This endeavor is really a study of diffusion processes. Loosely speaking, the term diffusion is attributed to a Markov process which has continuous sample paths and can be characterized in terms of its infinitesimal generator.

Stochastic Differential Equations

1. Introduction. 2. Rudiments of Linear Algebra and Multivariate Normal Theory. 3. Sufficiency and MVUB Estimators. 4. Neyman-Pearson Detectors. 5. Bayes Detectors. 6. Maximum Likelihood Estimators. 7. Bayes Estimators. 8. Minimum Mean-Squared Error Estimators. 9. Least Squares. 10. Linear Prediction. 11. Modal Analysis.

Statistical signal processing : detection, estimation, and time series analysis

We formulate a general class of problems for detecting subspace signals in subspace interference and broadband noise. We derive the generalized likelihood ratio (GLR) for each problem in the class. We then establish the invariances for the GLR and argue that these are the natural invariances for the problem. In each case, the GLR is a maximal invariant statistic, and the distribution of the maximal invariant statistic is monotone. This means that the GLR test (GLRT) is the uniformly most powerful invariant detector. We illustrate the utility of this finding by solving a number of problems for detecting subspace signals in subspace interference and broadband noise. In each case we give the distribution for the detector and compute performance curves. >

/pdf/matched-subspace-detectors-4jht34u3bq.pdf

Matched subspace detectors

Prony analysis extends Fourier analysis by directly estimating the frequency, damping, strength, and relative phase of modal components present in a given signal. The ability to extract such information from transient stability program simulations and from large-scale system tests of disturbances would be quite valuable to power system engineers. Early results of the application of this method to stability program output are reported. Also included are benchmarks against known models and a brief mathematical summary. >

/pdf/initial-results-in-prony-analysis-of-power-system-response-3sqn17b96s.pdf

Initial results in Prony analysis of power system response signals

The Wiener filter is analyzed for stationary complex Gaussian signals from an information theoretic point of view. A dual-port analysis of the Wiener filter leads to a decomposition based on orthogonal projections and results in a new multistage method for implementing the Wiener filter using a nested chain of scalar Wiener filters. This new representation of the Wiener filter provides the capability to perform an information-theoretic analysis of previous, basis-dependent, reduced-rank Wiener filters. This analysis demonstrates that the cross-spectral metric is optimal in the sense that it maximizes mutual information between the observed and desired processes. A new reduced-rank Wiener filter is developed based on this new structure which evolves a basis using successive projections of the desired signal onto orthogonal, lower dimensional subspaces. The performance is evaluated using a comparative computer analysis model and it is demonstrated that the low-complexity multistage reduced-rank Wiener filter is capable of outperforming the more complex eigendecomposition-based methods.

/pdf/a-multistage-representation-of-the-wiener-filter-based-on-38egqe5yyp.pdf

A multistage representation of the Wiener filter based on orthogonal projections

The theory of compressed sensing suggests that successful inversion of an image of the physical world (broadly defined to include speech signals, radar/sonar returns, vibration records, sensor array snapshot vectors, 2-D images, and so on) for its source modes and amplitudes can be achieved at measurement dimensions far lower than what might be expected from the classical theories of spectrum or modal analysis, provided that the image is sparse in an apriori known basis. For imaging problems in spectrum analysis, and passive and active radar/sonar, this basis is usually taken to be a DFT basis. However, in reality no physical field is sparse in the DFT basis or in any apriori known basis. No matter how finely we grid the parameter space the sources may not lie in the center of the grid cells and consequently there is mismatch between the assumed and the actual bases for sparsity. In this paper, we study the sensitivity of compressed sensing to mismatch between the assumed and the actual sparsity bases. We start by analyzing the effect of basis mismatch on the best k-term approximation error, which is central to providing exact sparse recovery guarantees. We establish achievable bounds for the l1 error of the best k -term approximation and show that these bounds grow linearly with the image (or grid) dimension and the mismatch level between the assumed and actual bases for sparsity. We then derive bounds, with similar growth behavior, for the basis pursuit l1 recovery error, indicating that the sparse recovery may suffer large errors in the presence of basis mismatch. Although, we present our results in the context of basis pursuit, our analysis applies to any sparse recovery principle that relies on the accuracy of best k-term approximations for its performance guarantees. We particularly highlight the problematic nature of basis mismatch in Fourier imaging, where spillage from off-grid DFT components turns a sparse representation into an incompressible one. We substantiate our mathematical analysis by numerical examples that demonstrate a considerable performance degradation for image inversion from compressed sensing measurements in the presence of basis mismatch, for problem sizes common to radar and sonar.

/pdf/sensitivity-to-basis-mismatch-in-compressed-sensing-4f52mixbp7.pdf

Louis L. Scharf

Papers

Statistical signal processing : detection, estimation, and time series analysis

Matched subspace detectors

Initial results in Prony analysis of power system response signals

A multistage representation of the Wiener filter based on orthogonal projections

Sensitivity to Basis Mismatch in Compressed Sensing