520 IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 56, NO. 1, JANUARY 2010
Beyond Nyquist: Efficient Sampling of Sparse
Bandlimited Signals
Joel A. Tropp, Member, IEEE, Jason N. Laska, Student Member, IEEE, Marco F. Duarte, Member, IEEE,
Justin K. Romberg, Member, IEEE, and Richard G. Baraniuk, Fellow, IEEE
Dedicated to the memory of Dennis M. Healy
Abstract—Wideband analog signals push contemporary analog-
to-digital conversion (ADC) systems to their performance limits. In
many applications, however, sampling at the Nyquist rate is ineffi-
cient because the signals of interest contain only a small number of
significant frequencies relative to the band limit, although the lo-
cations of the frequencies may not be known a priori. For this type
of sparse signal, other sampling strategies are possible. This paper
describes a new type of data acquisition system, called a random de-
modulator, that is constructed from robust, readily available com-
ponents. Let
K
denote the total number of frequencies in the signal,
and let
W
denote its band limit in hertz. Simulations suggest that
the random demodulator requires just
O(
K
log(
W=K
))
samples
per second to stably reconstruct the signal. This sampling rate is
exponentially lower than the Nyquist rate of
W
hertz. In contrast to
Nyquist sampling, one must use nonlinear methods, such as convex
programming, to recover the signal from the samples taken by the
random demodulator. This paper provides a detailed theoretical
analysis of the system’s performance that supports the empirical
observations.
Index Terms—Analog-to-digital conversion, compressive sam-
pling, sampling theory, signal recovery, sparse approximation.
I. INTRODUCTION
T
HE Shannon sampling theorem is one of the foundations
of modern signal processing. For a continuous-time signal
whose highest frequency is less than hertz, the theorem
Manuscript received January 31, 2009; revised September 18, 2009. Cur-
rent version published December 23, 2009. The work of J. A. Tropp was
supported by ONR under Grant N00014-08-1-0883, DARPA/ONR under
Grants N66001-06-1-2011 and N66001-08-1-2065, and NSF under Grant
DMS-0503299. The work of J. N. Laska, M. F. Duarte, and R. G. Bara-
niuk was supported by DARPA/ONR under Grants N66001-06-1-2011 and
N66001-08-1-2065, ONR under Grant N00014-07-1-0936, AFOSR under
Grant FA9550-04-1-0148, NSF under Grant CCF-0431150, and the Texas
Instruments Leadership University Program. The work of J. K. Romberg was
supported by NSF under Grant CCF-515632. The material in this paper was
presented in part at SampTA 2007, Thessaloniki, Greece, June 2007.
J. A. Tropp is with California Institute of Technology, Pasadena, CA 91125
USA (e-mail: jtropp@acm.caltech.edu).
J. N. Laska and R. G. Baraniuk are with Rice University, Houston, TX 77005
USA (e-mail: laska@rice.edu; richb@rice.edu).
M. F. Duarte was with Rice University, Houston, TX 77005 USA. He ia
now with Princeton University, Princeton, NJ 08554 USA (e-mail: mduarte@
princeton.edu).
J. K. Romberg is with the Georgia Institute of Technology, Atlanta GA 30332
USA (e-mail: jrom@ece.gatech.edu).
Communicated by H. Bölcskei, Associate Editor for Detection and Estima-
tion.
Color versions of Figures 2–8 in this paper are available online at http://iee-
explore.ieee.org.
Digital Object Identifier 10.1109/TIT.2009.2034811
suggests that we sample the signal uniformly at a rate of
hertz. The values of the signal at intermediate points in time are
determined completely by the cardinal series
(1)
In practice, one typically samples the signal at a somewhat
higher rate and reconstructs with a kernel that decays faster
than the
function [1, Ch. 4].
This well-known approach becomes impractical when the
band limit
is large because it is challenging to build sam-
pling hardware that operates at a sufficient rate. The demands
of many modern applications already exceed the capabilities
of current technology. Even though recent developments in
analog-to-digital converter (ADC) technologies have increased
sampling speeds, state-of-the-art architectures are not yet ad-
equate for emerging applications, such as ultra-wideband and
radar systems because of the additional requirements on power
consumption [2]. The time has come to explore alternative
techniques [3].
A. The Random Demodulator
In the absence of extra information, Nyquist-rate sampling
is essentially optimal for bandlimited signals [4]. Therefore,
we must identify other properties that can provide additional
leverage. Fortunately, in many applications, signals are also
sparse. That is, the number of significant frequency compo-
nents is often much smaller than the band limit allows. We can
exploit this fact to design new kinds of sampling hardware.
This paper studies the performance of a new type of sam-
pling system—called a random demodulator—that can be used
to acquire sparse, bandlimited signals. Fig. 1 displays a block
diagram for the system, and Fig. 2 describes the intuition be-
hind the design. In summary, we demodulate the signal by mul-
tiplying it with a high-rate pseudonoise sequence, which smears
the tones across the entire spectrum. Then we apply a low-pass
antialiasing filter, and we capture the signal by sampling it at
a relatively low rate. As illustrated in Fig. 3, the demodulation
process ensures that each tone has a distinct signature within
the passband of the filter. Since there are few tones present, it
is possible to identify the tones and their amplitudes from the
low-rate samples.
The major advantage of the random demodulator is that it by-
passes the need for a high-rate ADC. Demodulation is typically
much easier to implement than sampling, yet it allows us to use
0018-9448/$26.00 © 2009 IEEE
Authorized licensed use limited to: CALIFORNIA INSTITUTE OF TECHNOLOGY. Downloaded on January 15, 2010 at 17:24 from IEEE Xplore. Restrictions apply.
TROPP et al.: EFFICIENT SAMPLING OF SPARSE BANDLIMITED SIGNALS 521
Fig. 1. Block diagram for the random demodulator. The components include a
random number generator, a mixer, an accumulator, and a sampler.
Fig. 2. Action of the demodulator on a pure tone. The demodulation process
multiplies the continuous-time input signal by a random square wave. The action
of the system on a single tone is illustrated in the time domain (left) and the
frequency domain (right). The dashed line indicates the frequency response of
the lowpass filter. See Fig. 3 for an enlargement of the filter’s passband.
Fig. 3. Signatures of two different tones. The random demodulator furnishes
each frequency with a unique signature that can be discerned by examining the
passband of the antialiasing filter. This image enlarges the pass region of the
demodulator’s output for two input tones (solid and dashed). The two signatures
are nearly orthogonal when their phases are taken into account.
a low-rate ADC. As a result, the system can be constructed from
robust, low-power, readily available components even while it
can acquire higher band limit signals than traditional sampling
hardware.
We do pay a price for the slower sampling rate: It is no longer
possible to express the original signal
as a linear function of
the samples, àlathe cardinal series (1). Rather,
is encoded
into the measurements in a more subtle manner. The reconstruc-
tion process is highly nonlinear, and must carefully take advan-
tage of the fact that the signal is sparse. As a result, signal re-
covery becomes more computationally intensive. In short, the
random demodulator uses additional digital processing to re-
duce the burden on the analog hardware. This tradeoff seems ac-
ceptable, as advances in digital computing have outpaced those
in ADC.
B. Results
Our simulations provide striking evidence that the random
demodulator performs. Consider a periodic signal with a
band limit of
hertz, and suppose that it contains
tones with random frequencies and phases. Our experiments
below show that, with high probability, the system acquires
enough information to reconstruct the signal after sampling
at just
hertz. In words, the sampling rate
is proportional to the number
of tones and the logarithm
of the bandwidth
. In contrast, the usual approach requires
sampling at
hertz, regardless of . In other words, the
random demodulator operates at an exponentially slower sam-
pling rate! We also demonstrate that the system is effective for
reconstructing simple communication signals.
Our theoretical work supports these empirical conclusions,
but it results in slightly weaker bounds on the sampling rate. We
have been able to prove that a sampling rate of
suffices for high-probability recovery of the random
signals we studied experimentally. This analysis also suggests
that there is a small startup cost when the number of tones is
small, but we did not observe this phenomenon in our experi-
ments. It remains an open problem to explain the computational
results in complete detail.
The random signal model arises naturally in numerical exper-
iments, but it does not provide an adequate description of real
signals, whose frequencies and phases are typically far from
random. To address this concern, we have established that the
random demodulator can acquire all
-tone signals—regard-
less of the frequencies, amplitudes, and phases—when the sam-
pling rate is
. In fact, the system does not even
require the spectrum of the input signal to be sparse; the system
can successfully recover any signal whose spectrum is well-ap-
proximated by
tones. Moreover, our analysis shows that the
random demodulator is robust against noise and quantization
errors.
This work focuses on input signals drawn from a specific
mathematical model, framed in Section II. Many real signals
have sparse spectral occupancy, even though they do not meet
all of our formal assumptions. We propose a device, based on
the classical idea of windowing, that allows us to approximate
general signals by signals drawn from our model. Therefore, our
recovery results for the idealized signal class extend to signals
that we are likely to encounter in practice.
In summary, we believe that these empirical and theoretical
results, taken together, provide compelling evidence that the de-
modulator system is a powerful alternative to Nyquist-rate sam-
pling for sparse signals.
C. Outline
In Section II, we present a mathematical model for the class
of sparse, bandlimited signals. Section III describes the intu-
ition and architecture of the random demodulator, and it ad-
dresses the nonidealities that may affect its performance. In
Section IV, we model the action of the random demodulator as
a matrix. Section V describes computational algorithms for re-
constructing frequency-sparse signals from the coded samples
provided by the demodulator. We continue with an empirical
study of the system in Section VI, and we offer some theoret-
ical results in Section VII that partially explain the system’s per-
formance. Section VIII discusses a windowing technique that
allows the demodulator to capture nonperiodic signals. We con-
clude with a discussion of potential technological impact and
Authorized licensed use limited to: CALIFORNIA INSTITUTE OF TECHNOLOGY. Downloaded on January 15, 2010 at 17:24 from IEEE Xplore. Restrictions apply.
522 IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 56, NO. 1, JANUARY 2010
related work in Sections IX and X. Appendices I–III contain
proofs of our signal reconstruction theorems.
II. T
HE
SIGNAL MODEL
Our analysis focuses on a class of discrete, multitone signals
that have three distinguished properties.
•
Bandlimited. The maximum frequency is bounded.
• Periodic. Each tone has an integral frequency in hertz.
• Sparse. The number of active tones is small in comparison
with the band limit.
Our work shows that the random demodulator can recover these
signals very efficiently. Indeed, the number of samples per unit
time scales directly with the sparsity, but it increases only loga-
rithmically in the band limit.
At first, these discrete multitone signals may appear simpler
than the signals that arise in most applications. For example, we
often encounter signals that contain nonharmonic tones or sig-
nals that contain continuous bands of active frequencies rather
than discrete tones. Nevertheless, these broader signal classes
can be approximated within our model by means of
windowing
techniques. We address this point in Section VIII.
A. Mathematical Model
Consider the following mathematical model for a class of
discrete multitone signals. Let
be a positive integer that
exceeds the highest frequency present in the continuous-time
signal
. Fix a number that represents the number of active
tones. The model contains each signal of the form
for (2)
Here,
is a set of integer-valued frequencies that satisfies
and
is a set of complex-valued amplitudes. We focus on the case
where the number
of active tones is much smaller than the
bandwidth
.
To summarize, the signals of interest are bandlimited because
they contain no frequencies above
cycles per second; pe-
riodic because the frequencies are integral; and sparse because
the number of tones
. Let us emphasize several con-
ceptual points about this model.
• We have normalized the time interval to one second for
simplicity. Of course, it is possible to consider signals at
another time resolution.
• We have also normalized frequencies. To consider signals
whose frequencies are drawn from a set equally spaced by
, we would change the effective band limit to .
• The model also applies to signals that are sparse and
bandlimited in a single time interval. It can be extended
to signals where the model (2) holds with a different
set of frequencies and amplitudes in each time interval
. These signals are sparse not in the
Fourier domain but rather in the short-time Fourier domain
[5, Ch. IV].
B. Information Content of Signals
According to the sampling theorem, we can identify signals
from the model (2) by sampling for one second at
hertz.
Yet these signals contain only
bits of
information. In consequence, it is reasonable to expect that we
can acquire these signals using only
digital samples.
Here is one way to establish the information bound. Stirling’s
approximation shows that there are about
ways to select distinct integers in the range
. Therefore, it takes bits to
encode the frequencies present in the signal. Each of the
amplitudes can be approximated with a fixed number of bits, so
the cost of storing the frequencies dominates.
C. Examples
There are many situations in signal processing where we en-
counter signals that are sparse or locally sparse in frequency.
Here are some basic examples.
• Communications signals, such as transmissions with a
frequency-hopping modulation scheme that switches a
sinusoidal carrier among many frequency channels ac-
cording to a predefined (often pseudorandom) sequence.
Other examples include transmissions with narrowband
modulation where the carrier frequency is unknown but
could lie anywhere in a wide bandwidth.
• Acoustic signals, such as musical signals where each note
consists of a dominant sinusoid with a progression of sev-
eral harmonic overtones.
• Slowly varying chirps, as used in radar and geophysics,
that slowly increase or decrease the frequency of a sinusoid
over time.
• Smooth signals that require only a few Fourier coefficients
to represent.
• Piecewise smooth signals that are differentiable except for
a small number of step discontinuities.
We also note several concrete applications where sparse
wideband signals are manifest. Surveillance systems may
acquire a broad swath of Fourier bandwidth that contains
only a few communications signals. Similarly, cognitive radio
applications rely on the fact that parts of the spectrum are not
occupied [6], so the random demodulator could be used to
perform spectrum sensing in certain settings. Additional poten-
tial applications include geophysical imaging, where two- and
three-dimensional seismic data can be modeled as piecewise
smooth (hence, sparse in a local Fourier representation) [7], as
well as radar and sonar imaging [8], [9].
III. T
HE RANDOM DEMODULATOR
This section describes the random demodulator system that
we propose for signal acquisition. We first discuss the intuition
behind the system and its design. Then we address some imple-
mentation issues and nonidealities that impact its performance.
A. Intuition
The random demodulator performs three basic actions:
demodulation, lowpass filtering, and low-rate sampling. Refer
back to Fig. 1 for the block diagram. In this section, we offer
Authorized licensed use limited to: CALIFORNIA INSTITUTE OF TECHNOLOGY. Downloaded on January 15, 2010 at 17:24 from IEEE Xplore. Restrictions apply.
TROPP et al.: EFFICIENT SAMPLING OF SPARSE BANDLIMITED SIGNALS 523
a short explanation of why this approach allows us to acquire
sparse signals.
Consider the problem of acquiring a single high-frequency
tone that lies within a wide spectral band. Evidently, a low-rate
sampler with an antialiasing filter is oblivious to any tone whose
frequency exceeds the passband of the filter. The random de-
modulator deals with the problem by smearing the tone across
the entire spectrum so that it leaves a signature that can be de-
tected by a low-rate sampler.
More precisely, the random demodulator forms a (periodic)
square wave that randomly alternates at or above the Nyquist
rate. This random signal is a sort of periodic approximation
to white noise. When we multiply a pure tone by this random
square wave, we simply translate the spectrum of the noise, as
documented in Fig. 2. The key point is that translates of the noise
spectrum look completely different from each other, even when
restricted to a narrow frequency band, which Fig. 3 illustrates.
Now consider what happens when we multiply a frequency-
sparse signal by the random square wave. In the frequency do-
main, we obtain a superposition of translates of the noise spec-
trum, one translate for each tone. Since the translates are so
distinct, each tone has its own signature. The original signal
contains few tones, so we can disentangle them by examining
a small slice of the spectrum of the demodulated signal.
To that end, we perform lowpass filtering to prevent aliasing,
and we sample with a low-rate ADC. This process results in
coded samples that contain a complete representation of the
original sparse signal. We discuss methods for decoding the
samples in Section V.
B. System Design
Let us present a more formal description of the random
demodulator shown in Fig. 1. The first two components im-
plement the demodulation process. The first piece is a random
number generator, which produces a discrete-time sequence
of numbers that take values with equal proba-
bility. We refer to this as the chipping sequence. The chipping
sequence is used to create a continuous-time demodulation
signal
via the formula
and
In words, the demodulation signal switches between the levels
randomly at the Nyquist rate of hertz. Next, the mixer
multiplies the continuous-time input
by the demodulation
signal
to obtain a continuous-time demodulated signal
Together these two steps smear the frequency spectrum of the
original signal via the convolution
See Fig. 2 for a visual.
The next two components behave the same way as a standard
ADC, which performs lowpass filtering to prevent aliasing and
then samples the signal. Here, the lowpass filter is simply an ac-
cumulator that sums the demodulated signal
for sec-
onds. The filtered signal is sampled instantaneously every
seconds to obtain a sequence of measurements. After each
sample is taken, the accumulator is reset. In summary
This approach is called integrate-and-dump sampling. Finally,
the samples are quantized to a finite precision. (In this work, we
do not model the final quantization step.)
The fundamental point here is that the sampling rate
is
much lower than the Nyquist rate
. We will see that de-
pends primarily on the number
of significant frequencies that
participate in the signal.
C. Implementation and Nonidealities
Any reasonable system for acquiring continuous-time signals
must be implementable in analog hardware. The system that we
propose is built from robust, readily available components. This
subsection briefly discusses some of the engineering issues.
In practice, we generate the chipping sequence with a pseudo-
random number generator. It is preferable to use pseudorandom
numbers for several reasons: they are easier to generate; they
are easier to store; and their structure can be exploited by dig-
ital algorithms. Many types of pseudorandom generators can be
fashioned from basic hardware components. For example, the
Mersenne twister [10] can be implemented with shift registers.
In some applications, it may suffice just to fix a chipping se-
quence in advance.
The performance of the random demodulator is unlikely to
suffer from the fact that the chipping sequence is not completely
random. We have been able to prove that if the chipping se-
quence consists of
-wise independent random variables (for
an appropriate value of
), then the demodulator still offers the
same guarantees. Alon et al. have demonstrated that shift regis-
ters can generate a related class of random variables [11].
The mixer must operate at the Nyquist rate
. Nevertheless,
the chipping sequence alternates between the levels
, so the
mixer only needs to reverse the polarity of the signal. It is rela-
tively easy to perform this step using inverters and multiplexers.
Most conventional mixers trade speed for linearity, i.e., fast tran-
sitions may result in incorrect products. Since the random de-
modulator only needs to reverse polarity of the signal, nonlin-
earity is not the primary nonideality. Instead, the bottleneck for
the speed of the mixer is the settling times of inverters and mul-
tiplexors, which determine the length of time it takes for the
output of the mixer to reach steady state.
The sampler can be implemented with an off-the-shelf ADC.
It suffers the same types of nonidealities as any ADC, including
thermal noise, aperture jitter, comparator ambiguity, and so forth
[12]. Since the random demodulator operates at a relatively low
sampling rate, we can use high-quality ADCs, which exhibit
fewer problems.
In practice, a high-fidelity integrator is not required. It suffices
to perform lowpass filtering before the samples are taken. It is
Authorized licensed use limited to: CALIFORNIA INSTITUTE OF TECHNOLOGY. Downloaded on January 15, 2010 at 17:24 from IEEE Xplore. Restrictions apply.
524 IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 56, NO. 1, JANUARY 2010
essential, however, that the impulse response of this filter can be
characterized very accurately.
The net effect of these nonidealities is much like the addi-
tion of noise to the signal. The signal reconstruction process is
very robust, so it performs well even in the presence of noise.
Nevertheless, we must emphasize that, as with any device that
employs mixed signal technologies, an end-to-end random de-
modulator system must be calibrated so that the digital algo-
rithms are aware of the nonidealities in the output of the analog
hardware.
IV. R
ANDOM
DEMODULATION IN
MATRIX
FORM
In the ideal case, the random demodulator is a linear system
that maps a continuous-time signal to a discrete sequence of
samples. To understand its performance, we prefer to express
the system in matrix form. We can then study its properties using
tools from matrix analysis and functional analysis.
A. Discrete-Time Representation of Signals
The first step is to find an appropriate discrete representation
for the space of continuous-time input signals. To that end, note
that each
-second block of the signal is multiplied by a
random sign. Then these blocks are aggregated, summed, and
sampled. Therefore, part of the time-averaging performed by the
accumulator commutes with the demodulation process. In other
words, we can average the input signal over blocks of duration
without affecting subsequent steps.
Fix a time instant of the form
for an integer . Let
denote the average value of the signal over a time interval
of length
starting at . Thus
(3)
with the convention that, for the frequency
, the bracketed
term equals
. Since , the bracket never equals
zero. Absorbing the brackets into the amplitude coefficients, we
obtain a discrete-time representation
of the signal
for
where
In particular, a continuous-time signal that involves only the fre-
quencies in
can be viewed as a discrete-time signal comprised
of the same frequencies. We refer to the complex vector
as an
amplitude vector, with the understanding that it contains phase
information as well.
The nonzero components of the length-
vector are listed
in the set
. We may now express the discrete-time signal as
a matrix–vector product. Define the
matrix
where
and
The matrix is a simply a permuted discrete Fourier transform
(DFT) matrix. In particular,
is unitary and its entries share the
magnitude
.
In summary, we can work with a discrete representation
of the input signal.
B. Action of the Demodulator
We view the random demodulator as a linear system acting
on the discrete form
of the continuous-time signal .
First, we consider the effect of random demodulation on the
discrete-time signal. Let
be the chipping se-
quence. The demodulation step multiplies each
, which is
the average of
on the th time interval, by the random sign
. Therefore, demodulation corresponds to the map
where
.
.
.
is a diagonal matrix.
Next, we consider the action of the accumulate-and-dump
sampler. Suppose that the sampling rate is
, and assume that
divides . Then each sample is the sum of consecutive
entries of the demodulated signal. Therefore, the action of the
sampler can be treated as an
matrix whose th row
has
consecutive unit entries starting in column
for each . An example with and
is
When does not divide , two samples may share contribu-
tions from a single element of the chipping sequence. We choose
to address this situation by allowing the matrix
to have frac-
tional elements in some of its columns. An example with
and is
We have found that this device provides an adequate approxi-
mation to the true action of the system. For some applications,
it may be necessary to exercise more care.
In summary, the matrix
describes the action of
the hardware system on the discrete signal
. Each row of the
matrix yields a separate sample of the input signal.
The matrix
describes the overall action of the
system on the vector
of amplitudes. This matrix has a special
Authorized licensed use limited to: CALIFORNIA INSTITUTE OF TECHNOLOGY. Downloaded on January 15, 2010 at 17:24 from IEEE Xplore. Restrictions apply.
