Data Mining Practical Machine Learning Tools and Techniques

An overview of the self-organizing map algorithm, on which the papers in this issue are based, is presented in this article.

The Self-Organizing Map

1. Basic Concepts. 2. Nonparametric Methods. 3. Parametric Methods for Rational Spectra. 4. Parametric Methods for Line Spectra. 5. Filter Bank Methods. 6. Spatial Methods. Appendix A: Linear Algebra and Matrix Analysis Tools. Appendix B: Cramer-Rao Bound Tools. Appendix C: Model Order Selection Tools. Appendix D: Answers to Selected Exercises. Bibliography. References Grouped by Subject. Subject Index.

/pdf/spectral-analysis-of-signals-4dbupf9m4n.pdf

Spectral analysis of signals

The performance of the MUSIC and ML methods is studied, and their statistical efficiency is analyzed. The Cramer-Rao bound (CRB) for the estimation problems is derived, and some useful properties of the CRB covariance matrix are established. The relationship between the MUSIC and ML estimators is investigated as well. A numerical study is reported of the statistical efficiency of the MUSIC estimator for the problem of finding the directions of two plane waves using a uniform linear array. An exact description of the results is included. >

MUSIC, maximum likelihood, and Cramer-Rao bound

We explore in this chapter questions of existence and uniqueness for solutions to stochastic differential equations and offer a study of their properties. This endeavor is really a study of diffusion processes. Loosely speaking, the term diffusion is attributed to a Markov process which has continuous sample paths and can be characterized in terms of its infinitesimal generator.

Stochastic Differential Equations

Complex-valued signals occur in many areas of science and engineering and are thus of fundamental interest. In the past, it has often been assumed, usually implicitly, that complex random signals are proper or circular. A proper complex random variable is uncorrelated with its complex conjugate, and a circular complex random variable has a probability distribution that is invariant under rotation in the complex plane. While these assumptions are convenient because they simplify computations, there are many cases where proper and circular random signals are very poor models of the underlying physics. When taking impropriety and noncircularity into account, the right type of processing can provide significant performance gains. There are two key ingredients in the statistical signal processing of complex-valued data: 1) utilizing the complete statistical characterization of complex-valued random signals; and 2) the optimization of real-valued cost functions with respect to complex parameters. In this overview article, we review the necessary tools, among which are widely linear transformations, augmented statistical descriptions, and Wirtinger calculus. We also present some selected recent developments in the field of complex-valued signal processing, addressing the topics of model selection, filtering, and source separation.

/pdf/complex-valued-signal-processing-the-proper-way-to-deal-with-2dmglvcg5d.pdf

Complex-Valued Signal Processing: The Proper Way to Deal With Impropriety

We show that the Adaptive Coherence Estimator (ACE) is a uniformly most powerful (UMP) invariant detection statistic. This statistic is relevant to a scenario appearing in adaptive array processing, in which there are auxiliary, signal-free, training-data vectors that can be used to form a sample covariance estimate for clutter and interference suppression. The result is based on earlier work by Bose and Steinhardt, who found a two-dimensional (2-D) maximal invariant when test and training data share the same noise covariance. Their 2-D maximal invariant is given by Kelly's Generalized Likelihood Ratio Test (GLRT) statistic and the Adaptive Matched Filter (AMF). We extend the maximal-invariant framework to the problem for which the ACE is a GLRT: The test data shares the same covariance structure as the training data, but the relative power level is not constrained. In this case, the maximal invariant statistic collapses to a one-dimensional (1-D) scalar, which is also the ACE statistic. Furthermore, we show that the probability density function for the ACE possesses the property of "total positivity," which establishes that it has monotone likelihood ratio. Thus, a threshold test on the ACE is UMP-invariant. This means that it has a claim to optimality, having the largest detection probability out of the class of detectors that are also invariant to affine transformations on the data matrix that leave the hypotheses unchanged. This requires an additional invariance not imposed by Bose and Steinhardt: invariance to relative scaling of test and training data. The ACE is invariant and has a Constant False Alarm Rate (CFAR) with respect to such scaling, whereas Kelly's GLRT and the AMF are invariant, and CFAR, only with respect to common scaling.

The adaptive coherence estimator: a uniformly most-powerful-invariant adaptive detection statistic

Compressed sensing theory suggests that successful inversion of an image of the physical world from its modal parameters can be achieved at measurement dimensions far lower than the image dimension, provided that the image is sparse in an a priori known basis The assumed basis for sparsity typically corresponds to a gridding of the parameter space, eg, an DFT grid in spectrum analysis However, in reality no physical field is sparse in the DFT basis or in an a priori known basis No matter how finely we grid the parameter space the sources may not lie in the center of the grid cells and there is always mismatch between the assumed and the actual bases for sparsity In this paper, we study the sensitivity of compressed sensing (basis pursuit to be exact) to mismatch between the assumed and the actual sparsity bases Our mathematical analysis and numerical examples show that the performance of basis pursuit degrades considerably in the presence of basis mismatch

/pdf/sensitivity-to-basis-mismatch-in-compressed-sensing-2l8u3zva29.pdf

Louis L. Scharf

Papers

Complex-Valued Signal Processing: The Proper Way to Deal With Impropriety

The adaptive coherence estimator: a uniformly most-powerful-invariant adaptive detection statistic

Sensitivity to basis mismatch in compressed sensing

CFAR adaptive subspace detector is a scale-invariant GLRT, The

The SVD and reduced rank signal processing