Comparison of methods for spectral estimation with interrupted data
TL;DR: Tests with contrived data records indicate that two algorithms are preferable, one when the gap length is less than 15% of the record, and the other for 20%-50% gaps.
Abstract: The problem considered is to match the periodogram (spectrum) of a real sampled data sequence when only the samples outside a gap are available: that is, when the samples in the gap are missing or corrupted. Different arguments lead to three reasonable estimation algorithms. Tests with contrived data records indicate that two of these algorithms are preferable, one when the gap length is less than 15% of the record, and the other for 20%-50% gaps. An algorithm based on an autoregressive model is found to have an estimate performance that is relatively independent of gap length. >
Citations
More filters
TL;DR: In New Zealand, the historical tide gauge records provide a source of data for the analysis of sea level variations, but in New Zealand the data are generally of low quality.
Abstract: Historical tide gauge records provide a source of data for the analysis of sea level variations, but in New Zealand the data are generally of low quality. Techniques have been developed for handlin...
13 citations
24 Apr 1998
TL;DR: In this paper, a method is proposed to find the power spectrum of signals with missing observations, where the missing observations were first found by the linear prediction method and then an AR model was assumed and the Burg algorithm was used to estimate the spectrum for each segment of the signal.
Abstract: A method is proposed to find the power spectrum of signals with missing observations. The missing observations were first found by the linear prediction method and then an AR model was assumed and the Burg algorithm was used to estimate the spectrum for each segment of the signal. The coefficients found by Burg's method were updated every time the next value of the missing signal was predicted. After the missing observations were found then the power spectra were estimated using the following three different approaches: (a) averaging in the time domain, (b) averaging the spectra of each segment, and (c) averaging the coefficients of the AR model.
5 citations
TL;DR: The author presents an algorithm for iterative gap filling through the use of an autoregressive model that is validated with a simulation study, and its benefits are demonstrated through application to measured data sets.
Abstract: Power law spectrums are frequently used to model complex natural processes. Estimation of power law behavior can be severely hampered by temporal gaps in measured data, which can occur frequently for data sets spanning many years. The author presents an algorithm for iterative gap filling through the use of an autoregressive model. This technique is validated with a simulation study, and its benefits are demonstrated through application to measured data sets.
2 citations
Cites background from "Comparison of methods for spectral ..."
...The spectral analysis community has addressed temporal gaps with a variety of techniques [10]: multitaper averaging of the available data [11], iterative estimation of autoregressive (AR)–movingaverage parameters [12], [13], and iterative extrapolation to fill in the missing data through band-limited or line spectral assumptions [14], [15]....
[...]
22 Mar 2007
TL;DR: This paper examines the effectiveness of the Kim method on those signals that have less pronounced spectral peaks and are more broadband in nature, and the performance of the estimator is quantified for these signals and compared to the performance attained with the narrow band signals.
Abstract: Measurements of signals recorded via digital recording equipment often contain blocks of missing data due to equipment errors, the nature of the process under observation, or physical limitations of the experiment or hardware. Therefore, it is of great interest to reconstruct these signals in terms of their time or frequency domain characteristics, or in this case their power spectra. Of particular interest here are those signals for which a parametric model is appropriate for describing their spectral content, notably auto regressive. The "Kim method" is an efficient algorithm for reconstruction of segmented autoregressive signals using an extrapolative method based on estimates of the autoregressive model parameters of the existing data. It has proven effectiveness for segmented autoregressive narrow-band data with pronounced peaks. This paper examines the effectiveness of the method on those signals that have less pronounced spectral peaks and are more broadband in nature. The performance of the estimator is quantified for these signals and compared to the performance attained with the narrow band signals.
Cites background from "Comparison of methods for spectral ..."
...Foy [5] addresses several of these methods in his comparison paper....
[...]
References
More filters
TL;DR: In this paper, a new algorithm is proposed for computing the transform of a band-limited function, which is a simple iteration involving only the fast Fourier transform (FFT), and it is shown that the effect of noise and the error due to aliasing can be controlled by early termination of the iteration.
Abstract: If only a segment of a function f (t) is given, then its Fourier spectrum F(\omega) is estimated either as the transform of the product of f(t) with a time-limited window w(t) , or by certain techniques based on various a priori assumptions. In the following, a new algorithm is proposed for computing the transform of a band-limited function. The algorithm is a simple iteration involving only the fast Fourier transform (FFT). The effect of noise and the error due to aliasing are determined and it is shown that they can be controlled by early termination of the iteration. The proposed method can also be used to extrapolate bandlimited functions.
1,034 citations
01 Jun 1973
TL;DR: In this article, the authors examined the relative merits of finite-duration impulse response (FIR) and infinite duration impulse response(IIR) digital filters as interpolation filters and showed that FIR filters are generally to be preferred for interpolation.
Abstract: In many digital signal precessing systems, e.g., vacoders, modulation systems, and digital waveform coding systems, it is necessary to alter the sampling rate of a digital signal Thus it is of considerable interest to examine the problem of interpolation of bandlimited signals from the viewpoint of digital signal processing. A frequency dmnain interpretation of the interpolation process, through which it is clear that interpolation is fundamentally a linear filtering process, is presented, An examination of the relative merits of finite duration impulse response (FIR) and infinite duration impulse response (IIR) digital filters as interpolation filters indicates that FIR filters are generally to be preferred for interpolation. It is shown that linear interpolation and classical polynomial interpolation correspond to the use of the FIR interpolation filter. The use of classical interpolation methods in signal processing applications is illustrated by a discussion of FIR interpolation filters derived from the Lagrange interpolation formula. The limitations of these filters lead us to a consideration of optimum FIR filters for interpolation that can be designed using linear programming techniques. Examples are presented to illustrate the significant improvements that are obtained using the optimum filters.
643 citations
01 Apr 1981
TL;DR: It is shown that by predistorting the signal (and later removing this predistortion) it is possible to achieve spectral extrapolation, to broaden the class of signals for which these algorithms achieve convergence, and to improve their performance in the presence of broad-band noise.
Abstract: This paper describes a rather broad class of iterative signal restoration techniques which can be applied to remove the effects of many different types of distortions. These techniques also allow for the incorporation of prior knowledge of the signal in terms of the specification of a constraint operator. Conditions for convergence of the iteration under various combinations of distortions and constraints are explored. Particular attention is given to the use of iterative restoration techniques for constrained deconvolution, when the distortion band-limits the signal and spectral extrapolation must be performed. It is shown that by predistorting the signal (and later removing this predistortion) it is possible to achieve spectral extrapolation, to broaden the class of signals for which these algorithms achieve convergence, and to improve their performance in the presence of broad-band noise.
465 citations
TL;DR: It is shown that many of the existing extrapolation algorithms for noiseless observations are unified under the criterion of minimum norm least squares (MNLS) extrapolation, and some new algorithms useful for extrapolation and spectral estimation of band-limited sequences in one and two dimensions are presented.
Abstract: In this paper we present some new algorithms useful for extrapolation and spectral estimation of band-limited sequences in one and two dimensions. First we show that many of the existing extrapolation algorithms for noiseless observations are unified under the criterion of minimum norm least squares (MNLS) extrapolation. For example, the iterative algorithms proposed in [2] and [8]-[10] are shown to be special cases of a one-step gradient algorithm which has linear convergence. Convergence and other numerical properties are improved by going to a conjugate gradient algorithm. For noisy observations, these algorithms could be extended by considering a mean-square extrapolation criterion which gives rise to a mean-square extrapolation filter and also to a recursive extrapolation filter. Examples and application of these methods are given. Extension of these algorithms is made for problems where the signal is known to be periodic. A new set of functions called the periodic-discrete prolate spheroidal sequences (P-DPSS), analogous to DPSS [21], [22], are introduced and their properties are studied. Finally, several of these algorithms are generalized to two dimensions and the relevant equations are given.
243 citations
TL;DR: It will be shown that the basic extrapolation operation is feasible for only a particular subset of the class of band-limited signals, and an efficient algorithmic method for achieving the desired extrapolation on this subset is presented.
Abstract: In this paper, the task of extrapolating a time-truncated version of a band-limited signal shall be considered. It will be shown that the basic extrapolation operation is feasible for only a particular subset of the class of band-limited signals (i.e., the operation is well-posed mathematically). An efficient algorithmic method for achieving the desired extrapolation on this subset is then presented. This algorithm is structured so that all necessary signal manipulations involve signals which are everywhere zero except possibly on a finite "observation time" set. As a consequence, its implementation is straightforward and can be carried out in real time. This is to be contrasted with many existing extrapolation algorithms which theoretically involve operations on signals that are nonzero for almost all values of time. Their numerical implementation thereby necessitates an error producing time-truncation and a resultant deleterious effect on the corresponding extrapolation. Using straightforward algebraic operations, a convenient one-step extrapolation procedure is next developed. This is noteworthy in that this procedure thereby enables one to effectively circumvent any potentially slow convergence rate difficulties which typically characterize extrapolation algorithms. The effectiveness of this one-step procedure is demonstrated by means of two examples.
150 citations