Sampling-50 years after Shannon
Summary (4 min read)
Introduction
- The emphasis is on regular sampling, where the grid is uniform.
- Recently, there has been strong revival of the subject, which was motivated by the intense activity taking place around wavelets (see [7], [35], [80], and [85]).
II. SHANNON’S SAMPLING THEOREM REVISITED
- Shannon’s sampling theory is applicable whenever the input function is bandlimited.
- The optimal choice is the ideal filter , which suppresses aliasing completely without introducing any distortion in the bandpass region.
- In 1941, the English mathematician Hardy, who was referring to the basis functions in Whittaker’s cardinal series (1), wrote: “It is odd that, although these functions occur repeatedly in analysis, especially in the theory of interpolation, it does not seem to have been remarked explicitly that they form an orthogonal system” [55].
- This orthonormality property greatly simplifies the implementation of the approximation process by which a function is projected onto .
- The conclusion of this section is that the traditional sampling paradigm with ideal prefiltering yields an approximation , which is the orthogonal projection of the input function onto (the space of band-limited functions).
B. Minimum Error Sampling
- Having defined their signal space, the next natural question is how to obtain the ’s in (9) such that the signal model is a faithful approximation of some input function .
- The optimal solution in the least squares sense is the orthogonal projection, which can be specified as (18) where the ’s are the dual basis functions of .
- This is very similar to (6), except that theanalysisandsynthesisfunctions ( and , respectively) are not identical—in general, the approximation problem is not decoupled.
- It also inherits the translation-invariant structure of the basis functions: .
- Similar to what has been said for the band-limited case [see (4)], the algorithm described by (18) has a straightforward signal-processing interpretation (see Fig. 2).
C. Consistent Sampling
- The authors have just seen how to design an optimal sampling system.
- Provided there exists such that , then there is a unique signal approximationin that is consistent with in the sense that (23).
- The projection property implies that the method can reconstruct perfectly any signal that is included in the reconstruction space; i.e., , .
- To model the sampling process, the authors take the analysis function to be (Dirac delta).
- The interpolating signal that results from this process is This solution can also be presented in a form closer to the traditional one [see (1)] (30) where is the interpolation function given by (31).
E. Equivalent Basis Functions
- So far, the authors have encountered three types of basis functions: the generic ones , the duals ( ), and the interpolating ones ( ).
- Interestingly, it has been shown that the interpolator and the orthonormal function associated with many families of function spaces (splines, Dubuc–Deslaurier [42], etc.) both converge to Shannon’s sinc interpolator as the order (to be defined in Section IV-B) tends to infinity [9], [10], [130].
- If one is only concerned with the signal values at the integers, then the so-called cardinal representation is the most adequate; here, the knowledge of the underlying basis function is only necessary when one wishes to discretize some continuous signal-processing operator (an example being the derivative) [122].
- Another interesting aspect is the time-frequency localization of the basis functions.
- Furthermore, by using this type of basis function in a wavelet decomposition (see Section V-A), it is possible to trade one type of resolution for the other [31], [129].
F. Sampling and Reproducing Kernel Hilbert Spaces
- The authors now establish the connection between what has been presented so far and Yao and Nashed’s general formulation of sampling using reproducing kernel Hilbert spaces (RKHS’s) [87], [148].
- The reproducing kernel for a given Hilbert spaceis unique; is is also symmetric .
- The authors now show how these can be constructed with the help of Theorem 2.
- The oblique projection operator (24) may therefore be written in a form similar to (18) Formally, this is equivalent to where the authors have now identified the projection kernel (37).
- Note that it is the inclusion constraint (33) that specifies the reprodcuing kernel in a unique fashion.
G. Extension to Higher Dimensions
- All results that have been presented carry over directly for the representation of multidimensional signals (or images) , provided that the sampling is performed on the cartesian grid .
- This generalization holds because their basic mathematical tool, Poisson’s summation formula , remains valid in dimensions.
- In practice, one often uses separable generating functions of the form.
- This greatly simplifies the implementation because all filtering operations are separable.
- The dual functions remain separable as well.
IV. CONTROLLING THE APPROXIMATION ERROR
- The projection interpretation of the sampling process that has just been presented has one big advantage: it does not require the band-limited hypothesis and is applicable for any function .
- For more general spline-like spaces , the situation is more complicated.
- This has led researchers in signal processing, who wanted a simple way to determine the critical sampling step, to develop an accurate error estimation technique which is entirely Fourier-based [20], [21].
- This recent method is easy to apply in practice and yet powerful enough to recover most classical error bounds; it will be described next.
- The key parameter for controlling the approximation error is the sampling step .
A. Calculating the Error in the Frequency Domain
- Let denote a linear approximation operator with sampling step .
- This corresponds to the general sampling system described in Fig. 2 and includes all the algorithms described so far.
- In other words,Q commutes with the shift operator by integer multiples ofT .
- For the standard Shannon paradigm, which uses ideal analysis and reconstruction filters, the authors find that ; this confirms the fact that the approximation error is entirely due to the out-of-band portion of the signal.
- The -approximation error of the operator defined by can be written as (44) where is a correction term negligible under most circumstances.
C. Comparison of Approximation Algorithms
- The leading constant in (45) for linear spline interpolation is ; this is more than a factor of two above the projection algorithms b) and c), which both achieve the smallest possible constant .
- More generally, it has been shown that the performances ofth order orthogonal and oblique projectors are asymptotically equivalent, provided that the analysis and synthesis functions are biorthogonal and that satisfies the partition of unity [123].
- Under those hypotheses, the leading constant in (45) is given by (46) where denotes the th derivative of the Fourier transform of .
- A simpler and more direct formula in terms of the refinement filter is available for wavelets [20].
D. Comparison of Approximation Spaces
- The other interesting issue that the authors may address using the above approximation results is the comparison of approxi- mation subspaces.
- As increases, the spline approximation converges to Shannon’s band-limited solution [130].
- The last three generating functions are of order and are members of the so-called Moms (maximum order minimum support) family; the cubic spline is the smoothest member while the O-Moms is the one with the smallest asymptotic constant (46), which explains its better performance.
- The state-of-the art interpolation method in image processing [91], [94], but the situation is likely to change.
- For a more classical perspective on sampling, the authors refer to Jerri’s excellent tutorial article, which gives a very comprehensive view of the subject up to the mid 1970’s [62].
B. Generalized (or Multichannel) Sampling
- In 1977, Papoulis introduced a powerful extension of Shannon’s sampling theory, showing that a band-limited signal could be reconstructed exactly from the samples of the response of linear shift-invariant systems sampled at 1 the reconstruction rate [90].
- If the measurements are performed in a structured manner, then the reconstruction process is simplified: for example, in the Papoulis framework, it is achieved by multivariate filtering [23], [83].
- Typical instances of generalized sampling are interlaced and derivative sampling [75], [149], both of which are special cases of Papoulis’ formulation.
- More recently, Papoulis’ theory has been generalized in several directions.
- While still remaining with band-limited functions, it was extended for multidimensional, as well as multicomponent signals [25], [101].
C. Finite Elements and Multiwavelets
- To obtain a signal representation that has the same sampling density as before, the multifunctions are translated by integer multiples of (48).
- In the more recent multiwavelet constructions, the multiscaling functions satisfy a vector two-scale relation—similar to (17)—that involves a matrix refinement filter instead of a scalar one [4], [45], [51], [56], [116].
- The situation is very much the same as in the scalar case, when the generating function is noninterpolating (see Section III-D).
- Given the equidistant samples (or measurements) of a signal , the expansion coefficients are usually obtained through an appropriate digital prefiltering procedure (analysis filterbank) [54], [140], [146].
- Because of the importance of the finite elements in engineering, the quality of this type of approximation has been studied thoroughly by approximation theorists [64], [73], [111].
D. Frames
- The notion of frame, which generalizes that of a basis, was introduced by Duffin and Schaffer [47]; it plays a crucial role in nonuniform sampling [12].
- The main difference with the Riesz basis condition (10) is that the frame definition allows for redundancy: there may be more template functions than are strictly necessary.
- Practically, the signal reconstruction is obtained from the solution of an overdetermined system of linear equations [92]; see also [34], for an iterative algorithm whenis close to .
- Irregular Sampling Irregular or nonuniform sampling constitutes another whole area of research that the authors mention here only briefly to make the connection with what has been presented.
- 2) Nonuniform Splines and Radial Basis Functions:.
VI. CONCLUSION
- Fifty years later, Shannon’s sampling theory is still alive and well.
- 4, APRIL 2000 communications including the Web, sound, television, photography, multimedia, medical imaging, etc.
- Many of the results reviewed in this paper have a potential for being useful in practice because they allow for a realistic modeling of the acquisition process and offer much more flexibility than the traditional band-limited framework.
- Last, the authors believe that the general unifying view of sampling that has emerged during the past decade is beneficial because it offers a common framework for understanding—and hopefully improving—many techniques that have been traditionally studied by separate communities.
- Areas that may benefit from these developments are analog-to-digital conversion, signal and image processing, interpolation, computer graphics, imaging, finite elements, wavelets, and approximation theory.
Did you find this useful? Give us your feedback
Citations
1,206 citations
1,090 citations
Cites background or methods from "Sampling-50 years after Shannon"
...A signal class that plays an important role in sampling theory are signals in SI spaces [151]–[154]....
[...]
...This model encompasses many signals used in communication and signal processing including bandlimited functions, splines [151], multiband signals [108], [109], [155], [156], and pulse amplitude modulation signals....
[...]
...Before describing the three cases, we first briefly introduce the notion of sampling in shift-invariant (SI) subspaces, which plays a key role in the development of standard (subspace) sampling theory [135], [151]....
[...]
966 citations
Cites background or methods from "Sampling-50 years after Shannon"
...Adapting our results to this context leads to new MMV recovery methods as well as equivalence conditions under which the entire set can be determined efficiently....
[...]
...The ith element of a vectorx is denoted byx(i)....
[...]
...U NION OF SUBSPACES...
[...]
818 citations
672 citations
Cites background from "Sampling-50 years after Shannon"
...in the Strang–Fix theory can be reformulated more quantitatively as follows: for sufficiently smooth functions, that is to say, functions for which is finite, the norm of the difference betweenand the approximation obtained from (22) is given by [253], [274], [276], [277]...
[...]
...For more detailed information, the reader is referred to mentioned papers as well as several reviews [222], [253]....
[...]
References
20,028 citations
17,693 citations
"Sampling-50 years after Shannon" refers background in this paper
...In wavelet theory, it corresponds to the number of vanishing moments of the analysis wavelet [35], [79], [115]; it also implies that the transfer function of the refinement filter in (17) can be factorized as ....
[...]
...The reader who wishes to learn more about wavelets is referred to the standard texts [35], [79], [115], [139]....
[...]
...below) that appears in the theory of the wavelet transform [35], [79], [115], [139]....
[...]
14,157 citations
10,258 citations
8,588 citations
"Sampling-50 years after Shannon" refers background in this paper
...These are properties that cannot be enforced simultaneously in the conventional wavelet framework, with the notable exception of the Haar basis [33]....
[...]
Related Papers (5)
Frequently Asked Questions (14)
Q2. What future works have the authors mentioned in the paper "Sampling—50 years after shannon" ?
The subject is far from being closed and its importance is most likely to grow in the future with the ever-increasing trend of replacing analog systems by digital ones ; typical application areas are 582 PROCEEDINGS OF THE IEEE, VOL. Many of the results reviewed in this paper have a potential for being useful in practice because they allow for a realistic modeling of the acquisition process and offer much more flexibility than the traditional band-limited framework. Last, the authors believe that the general unifying view of sampling that has emerged during the past decade is beneficial because it offers a common framework for understanding—and hopefully improving—many techniques that have been traditionally studied by separate communities. Areas that may benefit from these developments are analog-to-digital conversion, signal and image processing, interpolation, computer graphics, imaging, finite elements, wavelets, and approximation theory.
Q3. What are some recent applications of generalized sampling?
Recent applications of generalized sampling include motion-compensated deinterlacing of televison images [11], [121], and super-resolution [107], [138].
Q4. What is the way to obtain the expansion coefficients of a signal?
Given the equidistant samples (or measurements) of a signal , the expansion coefficients areusually obtained through an appropriate digital prefiltering procedure (analysis filterbank) [54], [140], [146].
Q5. What is the interesting generalization of (9)?
An interesting generalization of (9) is to consider generating functions instead of a single one; this corresponds to the finite element—or multiwavelet—framework.
Q6. What is the computational solution to the special case?
The computational solution takes the form of a multivariate filterbank and is compatible with Papoulis’ theory in the special case .
Q7. What is the advantage of the projection interpretation of the sampling process?
The projection interpretation of the sampling process that has just been presented has one big advantage: it does not require the band-limited hypothesis and is applicable for any function .
Q8. What is the way to suppress aliasing?
When this is not the case, the standard signal-processing practice is to apply a low-pass filter prior to sampling in order to suppress aliasing.
Q9. What is the graph of the error kernels for the least squares spline app?
8. This graph clearly shows that, for signals that are predominantly low-pass (i.e., with a frequency content within the Nyquist band), the error tends to be smaller for higher order splines.
Q10. What is the logical approach to the sampling theorem?
Having reinterpreted the sampling theorem from the more abstract perspective of Hilbert spaces and of projection operators, the authors can take the next logical step and generalize the approach to other classes of functions.
Q11. What is the main reason for the study of the error estimation technique?
This has led researchers in signal processing, who wanted a simple way to determine the critical sampling step, to develop an accurate error estimation technique which is entirely Fourier-based [20], [21].
Q12. What is the common symbol used in the theory of the wavelet transform?
Note that the orthogonalized version plays a special role in the theory of the wavelet transform [81]; it is commonly represented by the symbol .
Q13. What is the way to predict the loss of performance when an approximation algorithm is?
It is especially interesting to predict the loss of performance when an approximation algorithm such as (30) and (24) is used instead of the optimal least squares procedure (18).
Q14. How much decay does a polynomial spline provide?
The authors observe that a polynomial spline approximation of degree provides an asymptotic decay of 1 20 dB per decade, which is consistent with (45).