Probability Distributions.- Linear Models for Regression.- Linear Models for Classification.- Neural Networks.- Kernel Methods.- Sparse Kernel Machines.- Graphical Models.- Mixture Models and EM.- Approximate Inference.- Sampling Methods.- Continuous Latent Variables.- Sequential Data.- Combining Models.

Pattern Recognition and Machine Learning

1. Basic Concepts. 2. Nonparametric Methods. 3. Parametric Methods for Rational Spectra. 4. Parametric Methods for Line Spectra. 5. Filter Bank Methods. 6. Spatial Methods. Appendix A: Linear Algebra and Matrix Analysis Tools. Appendix B: Cramer-Rao Bound Tools. Appendix C: Model Order Selection Tools. Appendix D: Answers to Selected Exercises. Bibliography. References Grouped by Subject. Subject Index.

/pdf/spectral-analysis-of-signals-4dbupf9m4n.pdf

Spectral analysis of signals

System Identification I

: This thesis applies neural network feature selection techniques to multivariate time series data to improve prediction of a target time series. Two approaches to feature selection are used. First, a subset enumeration method is used to determine which financial indicators are most useful for aiding in prediction of the S&P 500 futures daily price. The candidate indicators evaluated include RSI, Stochastics and several moving averages. Results indicate that the Stochastics and RSI indicators result in better prediction results than the moving averages. The second approach to feature selection is calculation of individual saliency metrics. A new decision boundary-based individual saliency metric, and a classifier independent saliency metric are developed and tested. Ruck's saliency metric, the decision boundary based saliency metric, and the classifier independent saliency metric are compared for a data set consisting of the RSI and Stochastics indicators as well as delayed closing price values. The decision based metric and the Ruck metric results are similar, but the classifier independent metric agrees with neither of the other metrics. The nine most salient features, determined by the decision boundary based metric, are used to train a neural network and the results are presented and compared to other published results. (AN)

/pdf/nonlinear-time-series-analysis-1j3a35n4y3.pdf

Nonlinear Time Series Analysis.

Time-Frequency Analysis.

Periodic signals can be decomposed into sets of sinusoids having frequencies that are integer multiples of a fundamental frequency. The problem of finding such fundamental frequencies from noisy observations is important in many speech and audio applications, where it is commonly referred to as pitch estimation. These applications include analysis, compression, separation, enhancement, automatic transcription and many more. In this book, an introduction to pitch estimation is given and a number of statistical methods for pitch estimation are presented. The basic signal models and associated estimation theoretical bounds are introduced, and the properties of speech and audio signals are discussed and illustrated. The presented methods include both single- and multi-pitch estimators based on statistical approaches, like maximum likelihood and maximum a posteriori methods, filtering methods based on both static and optimal adaptive designs, and subspace methods based on the principles of subspace orthogonality and shift-invariance. The application of these methods to analysis of speech and audio signals is demonstrated using both real and synthetic signals, and their performance is assessed under various conditions and their properties discussed. Finally, the estimators are compared in terms of computational and statistical efficiency, generalizability and robustness. Table of Contents: Fundamentals / Statistical Methods / Filtering Methods / Subspace Methods / Amplitude Estimation

/pdf/multi-pitch-estimation-1kv5atu3ph.pdf

Multi-Pitch Estimation

This paper considers the problem of estimating the time delays and Doppler shifts of a known waveform received via several distinct paths by an array of antennas. The general maximum likelihood estimator is presented, and is shown to require a 2d-dimensional nonlinear minimization, where d is the number of received signal reflections. Two alternative solutions based on signal and noise subspace fitting are proposed, requiring only a d-dimensional minimization. In particular, we show how to decouple the required search into a two-step procedure, where the delays are estimated and the Dopplers solved for explicitly. Initial conditions for the time delay search can be obtained by applying generalizations of the MUSIC and ESPRIT algorithms, which are also outlined in the paper. Simulation examples are included to illustrate the algorithms' performance relative to the Cramer-Rao bound.

/pdf/subspace-based-estimation-of-time-delays-and-doppler-shifts-28zg86nlys.pdf

Subspace-based estimation of time delays and Doppler shifts

The problem of estimating the parameters of complex-valued sinusoidal signals (cisoids, for short) from data corrupted by colored noise occurs in many signal processing applications. We present a simple formula for the asymptotic (large-sample) Cramer-Rao bound (CRB) matrix associated with this problem. The maximum likelihood method (MLM), which estimates both the signal and noise parameters, attains the performance corresponding to the asymptotic CRB, as the sample length increases. More interestingly, we show that a computationally much simpler nonlinear least-squares method (NLSM), which estimates the signal parameters only, achieves the same performance in large samples.

Cisoid parameter estimation in the colored noise case: asymptotic Cramer-Rao bound, maximum likelihood, and nonlinear least-squares

This paper presents computationally efficient implementations for several recent algorithms based on the iterative adaptive approach (IAA) for uniformly sampled one- and two-dimensional data sets, considering both the complete data case, and the cases when the data sets are missing samples, either lacking arbitrary locations, or having gaps or periodically reoccurring gaps. By exploiting the method's inherent low displacement rank, together with the development of suitable Gohberg-Semencul representations, and the use of data dependent trigonometric polynomials, the proposed implementations are shown to offer a reduction of the necessary computational complexity by at least one order of magnitude. Numerical simulations together with theoretical complexity measures illustrate the achieved performance gain.

Andreas Jakobsson

Papers

Multi-Pitch Estimation

Subspace-based estimation of time delays and Doppler shifts

Cisoid parameter estimation in the colored noise case: asymptotic Cramer-Rao bound, maximum likelihood, and nonlinear least-squares

Efficient Implementation of Iterative Adaptive Approach Spectral Estimation Techniques

Matched-filterbank interpretation of some spectral estimators