This paper gives an exposition of linear prediction in the analysis of discrete signals The signal is modeled as a linear combination of its past values and present and past values of a hypothetical input to a system whose output is the given signal In the frequency domain, this is equivalent to modeling the signal spectrum by a pole-zero spectrum The major part of the paper is devoted to all-pole models The model parameters are obtained by a least squares analysis in the time domain Two methods result, depending on whether the signal is assumed to be stationary or nonstationary The same results are then derived in the frequency domain The resulting spectral matching formulation allows for the modeling of selected portions of a spectrum, for arbitrary spectral shaping in the frequency domain, and for the modeling of continuous as well as discrete spectra This also leads to a discussion of the advantages and disadvantages of the least squares error criterion A spectral interpretation is given to the normalized minimum prediction error Applications of the normalized error are given, including the determination of an "optimal" number of poles The use of linear prediction in data compression is reviewed For purposes of transmission, particular attention is given to the quantization and encoding of the reflection (or partial correlation) coefficients Finally, a brief introduction to pole-zero modeling is given

Linear prediction: A tutorial review

https://bio3d.colorado.edu/SerialEM/serialEMJSB.pdf

Automated electron microscope tomography using robust prediction of specimen movements.

A summary of many of the new techniques developed in the last two decades for spectrum analysis of discrete time series is presented in this tutorial. An examination of the underlying time series model assumed by each technique serves as the common basis for understanding the differences among the various spectrum analysis approaches. Techniques discussed include the classical periodogram, classical Blackman-Tukey, autoregressive (maximum entropy), moving average, autotegressive-moving average, maximum likelihood, Prony, and Pisarenko methods. A summary table in the text provides a concise overview for all methods, including key references and appropriate equations for computation of each spectral estimate.

Spectrum analysis&#8212;A modern perspective

In network approaches to psychopathology, disorders result from the causal interplay between symptoms (e.g., worry → insomnia → fatigue), possibly involving feedback loops (e.g., a person may engage in substance abuse to forget the problems that arose due to substance abuse). The present review examines methodologies suited to identify such symptom networks and discusses network analysis techniques that may be used to extract clinically and scientifically useful information from such networks (e.g., which symptom is most central in a person's network). The authors also show how network analysis techniques may be used to construct simulation models that mimic symptom dynamics. Network approaches naturally explain the limited success of traditional research strategies, which are typically based on the idea that symptoms are manifestations of some common underlying factor, while offering promising methodological alternatives. In addition, these techniques may offer possibilities to guide and evaluate therape...

http://www.psychosystems.org/files/Literature/borsboomcramerannualreview.pdf

Network Analysis: An Integrative Approach to the Structure of Psychopathology

We discuss the relations between maximum-entropy (MAXENT) and other methods of spectral analysis such as the Schuster, Blackman-Tukey, maximum-likelihood, Bayesian, and Autoregressive (AR, ARMA, or ARIMA) models, emphasizing that they are not in conflict, but rather are appropriate in different problems. We conclude that: 1) "Orthodox" sampling theory methods are useful in problems where we have a known model (sampling distribution) for the properties of the noise, but no appreciable prior information about the quantities being estimated. 2) MAXENT is optimal in problems where we have prior information about multiplicities, but no noise. 3) The full Bayesian solution includes both of these as special cases and is needed in problems where we have both prior information and noise. 4) AR models are in one sense a special case of MAXENT, but in another sense they are ubiquitous in all spectral analysis problems with discrete time series. 5) Empirical methods such as Blackman-Tukey, which do not invoke even a likelihood function, are useful in the preliminary, exploratory phase of a problem where our knowledge is sufficient to permit intuitive judgments about how to organize a calculation (smoothing, decimation, windows, prewhitening, padding with zeroes, etc.) but insufficient to set up a quantitative model which would do the proper things for us automatically and optimally.

On the rationale of maximum-entropy methods

In applied science, resolution has always been, and still is, an important issue. Since it is not unambiguously defined, it is interpreted in many ways. In this paper, which reviews the concept of optical resolution, a number of these interpretations are discussed. A discussion of resolution has to be preceded by a discussion of what is actually understood by an ‘‘optical image.’’ In a remarkable paper, Ronchi1 distinguished ethereal images, calculated images, and detected images. The term ethereal image was introduced only to represent the physical nature of the imaging phenomenon. As is customary in science in general, attempts have been made to give a mathematical representation of this phenomenon, both geometrically and algebraically. According to Ronchi, the images that have thus been calculated are mere mathematical constructions and should therefore be called calculated images. In the past, many approaches to the concept of resolution concerned these calculated images. This resulted in the so-called classical resolution criteria, such as Rayleigh’s criterion and the associated reciprocal bandwidth of the image. These criteria provide resolution limits that are determined solely by the calculated shape of the point-spread function associated with the imaging aperture and the wavelength of the light. From now on, they will be called classical resolution limits. Calculated images are by their very nature exactly describable by a mathematical model and thus noise free. Such images do not occur in practice. Therefore Ronchi stated that the resolution of detected images is much more important than the classical resolution, since it provides practical information about the imaging system employed. Hence one should consider primarily the resolution of detected images instead of that of calculated images. This means a necessary introduction of some new quantities of interest, such as the energy of the source and the sensitivity properties of the detector. Since Ronchi’s paper, further research on resolution— concerning detected images instead of calculated ones— has shown that in the end, resolution is limited by systematic and random errors resulting in an inadequacy of the description of the observations by the mathematical model chosen. This important conclusion was independently drawn by many researchers who were approaching the concept of resolution from different points of view, which will be discussed in the subsequent sections.

Resolution: a survey

Maximum entropy spectral analysis is a method for the estimation of power spectra with a higher resolution than can be obtained with conventional techniques. This is achieved by extrapolation of the autocorrelation function in such a way that the entropy of the corresponding probability density function is maximized in each step of the extrapolation. This correspondence also gives a simple interpretation of the method without entropy considerations.

Alternative interpretation of maximum entropy spectral analysis (Corresp.)

The gradient and Hessian are often used in analytical and numerical function optimisation complex valued parameter estimation problems. In a number of signal processing applications the function is a real function of complex variables. Then the optimisation is usually carried out with respect to the real and imaginary part of these variables; therefore, the gradient and Hessian concerned are real. The reason for this approach is to avoid difficulties with the definition and interpretation of the gradient and Hessian with respect to the complex variables. Definitions of a complex gradient and Hessian are proposed to solve these difficulties. The proposed and the real gradient and Hessian are fully compatible and are related by simple linear transformations. The results presented are an extension of a result by Brandwood (1983) concerning a complex gradient.

Complex gradient and Hessian

The multivariate complex normal distribution usually employed in the literature is a special case since certain restrictions have been imposed on the covariances of the real and imaginary parts of its variables. A more general distribution is proposed of which the usual distribution is shown to be a special case. >

/pdf/the-multivariate-complex-normal-distribution-a-hf8epvcfu8.pdf

The multivariate complex normal distribution-a generalization

An expression is derived for a Cramer-Rao lower bound on the variance of unbiased estimators of complex parameters. >

/pdf/a-cramer-rao-lower-bound-for-complex-parameters-2djlvbr7z3.pdf

A. van den Bos

Papers

Resolution: a survey

Alternative interpretation of maximum entropy spectral analysis (Corresp.)

Complex gradient and Hessian

The multivariate complex normal distribution-a generalization

A Cramer-Rao lower bound for complex parameters