Brigham Young University Brigham Young University
BYU ScholarsArchive BYU ScholarsArchive
Faculty Publications
1999-01-01
A cloud-removal algorithm for SSM/I data A cloud-removal algorithm for SSM/I data
David G. Long
david_long@byu.edu
Douglas L. Daum
Quinn P. Remund
Follow this and additional works at: https://scholarsarchive.byu.edu/facpub
Part of the Electrical and Computer Engineering Commons
Original Publication Citation Original Publication Citation
Long, D. G., Q. P. Remund, and D. L. Daum. "A Cloud-Removal Algorithm for SSM/I Data."
Geoscience and Remote Sensing, IEEE Transactions on 37.1 illustrated through simulation and
real data distribution analysis. The results show that the second-highest value algorithm is
biased high. MMA provides a more accurate brightness temperature estimate in areas of
atmospheric distortion, whil(TRUNCATED) (1999): 54-62
BYU ScholarsArchive Citation BYU ScholarsArchive Citation
Long, David G.; Daum, Douglas L.; and Remund, Quinn P., "A cloud-removal algorithm for SSM/I data"
(1999).
Faculty Publications
. 631.
https://scholarsarchive.byu.edu/facpub/631
This Peer-Reviewed Article is brought to you for free and open access by BYU ScholarsArchive. It has been
accepted for inclusion in Faculty Publications by an authorized administrator of BYU ScholarsArchive. For more
information, please contact ellen_amatangelo@byu.edu.
54 IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING, VOL. 37, NO. 1, JANUARY 1999
A Cloud-Removal Algorithm for SSM/I Data
David G. Long, Senior Member, IEEE, Quinn P. Remund, and Douglas L. Daum
Abstract—Microwave radiometers, while traditionally utilized
in atmospheric and oceanic studies, can also be used in land
surface applications. However, the problem of undesirable at-
mospheric effects caused by clouds and precipitation must be
addressed. In this paper, temporal composite surface bright-
ness images are generated from special sensor microwave/imager
(SSM/I) data with the aid of new algorithms to eliminate small-
scale distortion caused by clouds or precipitation. Mean, second-
highest value, modified maximum average (MMA), and hybrid
compositing algorithms are compared. The effectiveness of each
algorithm is illustrated through simulation and real data distri-
bution analysis. The results show that the second-highest value
algorithm is biased high. MMA provides a more accurate bright-
ness temperature estimate in areas of atmospheric distortion,
while the mean is superior in regions with little or no distortion.
A hybrid algorithm is developed that is a combination of MMA
and mean. It utilizes the strengths of both to create a superior
algorithm for regions with varying levels of distortion. Uses of
composite images produced by these algorithms include stud-
ies of vegetation change, land cover classification, and surface
parameter extraction.
Index Terms— Cloud removal, compositing, electromagnetic
atmospheric interference, microwave radiometry.
I. INTRODUCTION
M
ICROWAVE radiometers, such as the special sensor
microwave/imager (SSM/I) [5], [6] have wide applica-
tion in atmospheric remote sensing over the ocean and provide
essential inputs to numerical weather prediction models. SSM/I
data have also been used for land and ice studies, including
measurements of snow cover classification [4], soil and plant
moisture content [8], [15], atmospheric moisture over land
[10], land surface temperature [12], and mapping polar ice
[18].
Because the surface brightness observed by the SSM/I may
be adversely affected by spatial variations in the atmospheric
profile over the surface, algorithms for cloud removal have
been developed [1], [10]. In this paper, we compare several
new algorithms that generate cloud-free composite images
from multiple passes of the study region. Simulations to
determine the effectiveness of these algorithms are performed.
Actual SSM/I data are analyzed by exploring the effects of
compositing algorithms on the pixel surface brightness temper-
ature distributions. This paper is organized as follows. After a
brief background discussion in Section II, Section III discusses
the production of no-cloud composite images. Section IV
introduces the modified maximum average (MMA) and hybrid
Manuscript received November 18, 1996; revised October 27, 1997.
The authors are with Brigham Young University, Provo, UT 84602-4099
USA (e-mail: long@ee.byu.edu).
Publisher Item Identifier S 0196-2892(99)00024-8.
TABLE I
SSM/I C
HANNELS
algorithms. A simulation experiment to compare the cloud-
removal algorithms is presented in Section V. Section VI
discusses the analysis of actual SSM/I data. Finally, the
conclusions are given.
II. B
ACKGROUND
The SSM/I is a total-power, seven-channel, four-frequency
radiometer [5]. The channels are horizontal and vertical polar-
izations at 19.35, 37.0, and 85.5 GHz and vertical polarization
at 22.235 GHz. It utilizes an integrate-and-dump filter as
the antenna scans the ground track [7]. The 3-dB antenna
footprints range from about 15 to 70 km in the along-track
direction and 13 to 43 km in the cross-track direction (see
Table I). The 3-dB antenna footprints, which are different
for each frequency, generally have an elliptical shape on the
surface of the earth due to the elevation angle of the radiometer
[6].
The brightness temperatures observed by the SSM/I are a
function of the effective brightness temperature of the earth’s
surface and the emission, scattering, and attenuation of the
atmosphere. Because of the spatial and temporal variability of
the surface brightness, which is a function of the properties of
the soil and overlaying vegetation and their physical tempera-
tures, it is difficult to decompose the observed brightness into
its individual components. The most crucial factors affecting a
radiometric measurement, however, are the surface emissivity
and temperature, the vegetation canopy, and the atmospheric
conditions [19].
III. G
ENERATION OF CLOUD-FREE IMAGES
One of the challenges in mapping the surface brightness
from spaceborne radiometer data is atmospheric distortion.
Cloud cover and precipitation are two primary sources of this
distortion. Although cloud and rain cause little microwave
attenuation for frequencies less than 10 GHz, the higher
microwave frequencies of the SSM/I (19.35, 22.235, 37.0, and
85.0 GHz) show substantial atmospheric loss due to scattering
0196–2892/99$10.00 © 1999 IEEE
Authorized licensed use limited to: Brigham Young University. Downloaded on February 2, 2009 at 16:49 from IEEE Xplore. Restrictions apply.
LONG et al.: CLOUD-REMOVAL ALGORITHM FOR SSM/I DATA 55
Fig. 1. Individual SSM/I swath examples of temporal atmospheric distor-
tions. These images were created by assigning the closest measurement value
to each pixel. Columns (from left to right) correspond to 1992 Julian Days
245, 248, 261, and 264 of the passes. Rows (from top to bottom) indicate
SSM/I channels 19 V, 22 V, 27 V, and 85 V.
from hydrometeors and water vapor. Over the ocean, the
atmospheric signal is used to deduce cloud water content
from the change in brightness. For studies of the land surface,
however, these atmospheric effects may be unwanted [17].
Clouds and precipitation affect surface brightness measure-
ments in two ways. First, the cloud scattering nonuniformly
lowers the measured brightness temperature for all frequencies
of the SSM/I with higher frequencies progressively more
sensitive. The reduction in brightness temperature can be
confused with surface features. Second, the clouds attenuate
the polarization differences caused by the geometric or chem-
ical composition of different surface types. This prevents the
surface polarization difference from being used to discriminate
between vegetation types and/or standing water.
Fig. 1 illustrates examples of atmospherically distorted
brightness temperature images in a region of the Amazon
Basin for all vertical polarization SSM/I channels. These
images, like those in this paper, were generated by assigning
to each pixel covered by the swath the value of the nearest
measurement. Other single-pass imaging techniques can also
be used, e.g., [2] and [11]. The distortions are evident in
the temporal variation of surface brightness temperature in
different areas. Note that, as expected, the distortions are more
pronounced in the higher frequency channels. This follows the
trend of increased atmospheric scattering due to clouds and
precipitation with increasing frequency. The distortions of
pixel values can be as high as 60 K for the higher frequency
channels. These distortions can greatly hinder the application
of SSM/I data to land studies.
While multichannel- and/or multisensor-based algorithms
for cloud removal have been previously used (e.g., [4], [14],
[16], and [17]), we use a single-channel algorithm similar to
[1]. By using only single-frequency information to generate
a “cloud-free” image of the surface, we avoid introducing
spurious correlation between the channels. For example, since
each frequency has a different footprint size, using lower fre-
quency data to remove atmospheric distortion effects in higher
frequency channels may unnecessarily exclude undistorted
values in the higher frequency channels [1].
Our algorithm is based on the assumption that temporal
surface brightness variations over an area are caused by small-
scale atmospheric effects rather than temporal changes in the
surface brightness. Using multiple passes over the surface,
we generate a composite image that represents the effective
surface brightness temperature over a multiday period. The
composite image is generated from images created from each
descending pass, though ascending passes can also be used.
In the example data that follow, 20 days of descending pass
SSM/I data (September 1992) over South America are used.
During this period, each pixel is observed from five to ten
times. The value of the composite pixel is computed from this
ensemble. The study region is considered a worst-case exam-
ple, with frequent rain and distortion events occurring up to
several times during the compositing interval. For this region,
20 days offers a good balance between the number of undis-
torted measurements in the ensemble and temporal variations
due to seasonal radiometric surface response variations. This is
some what less than the 30 days used by previous investigators
[1], but it provides adequate results. Areas with less-frequent
distortion events may be able to use shorter periods.
Choudhury and Tucker [1] removed temporal atmospheric
distortion by using the second-highest pixel value from the
ensemble as the composite pixel value. Since the atmo-
spheric distortion generally lowers the brightness temperature
measurements over land, high pixel values have the least
atmospheric influence. They reason that, since the highest
value is often strongly influenced by noise or processing
artifacts, they used the second-highest pixel value.
Choosing the second-highest value is an example of a rank
order statistic [3]. Another rank order technique is the median
filter [9]. As an estimator, a rank order statistic is noise-
reducing, but it is sensitive to the underlying distribution of the
samples [13]. Thus, the second-highest value technique’s abil-
ity to reduce noise is strongly influenced by the measurement
distribution.
Unfortunately, the distribution function for the SSM/I data
is not precisely known and it is not possible to analytically
determine the estimator variance. However, it is known that
in the presence of atmospheric distortion over land, the distri-
bution is skewed low, while the desired estimation parameter
is the mode on the high end of the distribution [14]. This
strongly suggests that the rank order statistic needed for this
application is a value closer to the highest value than to the
median value. Given this insight, the second-highest value
method is a reasonable approach.
Authorized licensed use limited to: Brigham Young University. Downloaded on February 2, 2009 at 16:49 from IEEE Xplore. Restrictions apply.
56 IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING, VOL. 37, NO. 1, JANUARY 1999
IV. MMA ALGORITHM
In an effort to improve the performance of a cloud-removal
algorithm, an MMA technique was developed. This algorithm
attempts to estimate the cloud-free surface brightness of a pixel
by choosing a subset of pixel values from the ensemble of
measurements of that pixel and then averaging the selected
values together. By properly selecting the subset from the
ensemble, the cloud distortion is eliminated. Averaging of the
subset reduces the noise and attenuates any residual bias.
To select pixel values from the ensemble in the MMA
technique, the sample mean of the entire pixel ensemble is first
computed. Measurements greater than the sample mean yield a
subset of the complete ensemble corresponding to its highest
values. The highest value of this subset is then eliminated.
The remaining values consist of those values that are above
the ensemble mean but less than the maximum value of the
ensemble. This is the MMA subset. The estimated pixel value
is then determined as the mean of this subset.
Analyzing this technique statistically is challenging for two
reasons: 1) the distribution of pixel values when distortions are
included is not clearly known and 2) the algorithm combines
both box averaging statistics and order statistics. To qualita-
tively justify this approach, consider a simple model for the
pixel measurements. In this model, the measurement is the
sum of a Gaussian-distributed surface brightness temperature
and a weighted binary random variable
(1)
where
is the measured brightness temperature, is the
Gaussian distribution with mean
and standard deviation ,
is a binary-valued random variable of the probability that a
measurement contains cloud distortion (less than 30% based on
a simple examination of SSM/I in the study region described
later), and
is a positive random variable representing the
drop in brightness temperature due to a cloud (
depends on
the cloud thickness, water content, etc., the statistics of which
are unknown). A schematic example of the distribution of
is
given in Fig. 2(a), where
K and %. The right
mode corresponds to the distribution of surface brightness,
while the lower mode represents the distribution of cloudy
pixels. The “X” marks below the temperature axis illustrate
an ensemble of seven random measurements for a given pixel.
Also illustrated are the results from applying the MMA and
second-highest value techniques.
Good metrics for comparing estimation algorithms include
the mean estimate error (bias) and the estimate variance.
Ideally, the estimate should have no bias and minimum vari-
ance. To compare the variances of the MMA algorithm and
the second-highest value technique, consider Fig. 2(b). As in
Fig. 2(a), the X’s represent an ensemble of seven samples
taken from the distribution. The variance of the second-highest
value technique is governed by the average temperature dif-
ference between the highest and third highest value of the
ensemble. The variance of the MMA algorithm depends on
the variances of the second, third, and fourth measurements.
Graphically, we may see that the averaging of these values
lowers the estimate variance more than just using the second-
highest value.
(a)
(b)
Fig. 2. Illustration of hypothetical radiometric measurement distribution for
a cloudy region. (a) Sample discrete ensemble. (b) Variance of the modified
maximum and second-highest value.
Like the second-highest value estimate, the MMA estimate
in this example is biased high, and it is high whenever the
ensemble includes more than one sample from the lower mode
of the mixture distribution. However, it is clear that the MMA
bias is less than the second-highest value estimate. Further,
the estimator variance is smaller for MMA.
While MMA produces a less-biased estimate for pixels with
high cloud contamination than the second-highest value, it
is still biased high for pixels with little or no contamina-
tion. Fig. 3 depicts a hypothetical distribution of brightness
temperatures for a noncloud-affected pixel. In this case, the
second-highest value and MMA estimates are biased high.
The desired value is the mean of the overall distribution in
the absence of clouds or precipitation. As previously noted, a
simple examination of SSM/I data reveals a probability of less
than 30% that a measurement is distorted by clouds or rain.
Thus, MMA may be unnecessarily biased somewhat high for
many of the measurements.
In an effort to ameliorate this problem, a hybrid of the mean
and MMA methods has been developed. Ideally, this hybrid
implements MMA in the presence of clouds and takes the
mean in their absence. This reduces the overshoot of MMA
for low atmospheric distortions and provides a better estimate
of the actual surface brightness temperature.
To implement the hybrid algorithm, a metric is required for
the decision making process. The chosen metric is the temporal
standard deviation of the values for a particular pixel. The pres-
ence of clouds skews the brightness temperature distribution
low for affected passes, thus increasing the standard deviation.
Fig. 4 shows a mean SSM/I composite image along with
Authorized licensed use limited to: Brigham Young University. Downloaded on February 2, 2009 at 16:49 from IEEE Xplore. Restrictions apply.
LONG et al.: CLOUD-REMOVAL ALGORITHM FOR SSM/I DATA 57
(a)
(b)
Fig. 3. Illustration of hypothetical radiometric measurement distribution for
a clear region. (a) Sample discrete ensemble. (b) Variance of the modified
maximum and second-highest value.
its corresponding temporal standard deviation image for the
85-GHz vertically polarized channel. This visually illustrates
that the standard deviation highly correlates with atmospheric
distortions. Areas that appear darkened in the mean composite
image exhibit relatively high values in the standard deviation
image.
In the standard deviation image of Fig. 4, the areas with
low values correspond to regions with little or no atmospheric
distortion. A small 2
2 spatially homogeneous subregion,
which will be more explicitly defined in a later section, is
chosen as an example of an area with low temporal variation
and thus low atmospheric distortion. The temporal mean and
standard deviation of all swath pixel values are calculated
for each vertically polarized SSM/I channel in this subregion
and shown in Table II. The standard deviations represent the
temporal variance of surface brightness temperature in the
absence of atmospheric distortion. According to the previous
discussion, any kind of temporal variation, such as atmospheric
distortion, will cause the standard deviation to rise above these
values. All channel standard deviation values are similar with
the 19-V channel exhibiting the highest and the 37-V channel
the lowest. Ideally, optimum values should be used for each
channel in implementing the hybrid algorithm. However, since
the temporal standard deviations are similar, and for the sake
of simplicity, we chose to use the highest of these values
1.25 K as the hybrid threshold metric for the results presented
in this paper. In the hybrid algorithm, the standard deviation is
computed for each pixel ensemble of brightness temperatures.
If it is above 1.25 K, the MMA algorithm is used to produce
the composite value for that particular pixel to select only
nondistorted measurements. Otherwise, the mean is used. We
note that this threshold has been chosen for use in the study
region and should be tuned for other regions.
V. S
IMULATION
To further compare and contrast the mean, second-highest
value, MMA, and hybrid algorithms, a simple Monte Carlo
analysis for a single pixel is presented. This simulation as-
sumes that the true pixel brightness for a geographical area is
280 K. An ensemble of seven pixel values is then created by
adding a Gaussian random variable of standard deviation 1 K
to the “true” value. This represents the radiometric “noise”
inherent to the radiometer measurements. Seven pixels
simulate the average number of radiometric measurements in
the 20-day study period. Two of the ensemble measurements
then have simulated atmospheric distortion added. The first
measurement is reduced by
and the second measurement
by
2. This models a pixel that is contaminated by clouds
at two different times with one cloud twice as distorting as
the other. The seven-member ensemble is then processed by
each algorithm, and the results are saved. The results of 1000
simulations are then analyzed to give the results in Fig. 5.
For comparison, the ensemble mean is plotted along with the
windowed mean. The windowed mean is the mean of values
within one standard deviation of the ensemble mean.
For pixels with little or no atmospheric distortion, the mean
or windowed average is closer to the 280 “true” value than
MMA or the second-highest value. For ensembles that have
greater (
5 K) atmospheric distortion, the second-highest
value and MMA techniques are superior. The MMA tech-
nique has the smallest bias of the two. Since it also has the
smallest variance, the MMA algorithm is considered superior
to the second-highest value algorithm. The hybrid algorithm
combines the strengths of the mean method for low distortion
temperatures and MMA for high distortion temperatures. This
is evident in Fig. 5 by the closer estimates to 280 K for small
. The simulation results indicate that MMA is superior
in the presence of significant distortion and mean is best
with little or no distortion present, while the hybrid algorithm
combines the two in a manner that uses the appropriate
algorithm for each pixel.
VI. SSM/I D
ATA ANALYSIS
To validate the algorithms with actual data, a region of
the Amazon Basin was chosen for SSM/I data analysis. The
region lies primarily within Brazil and is bounded by the
coordinates: 48–63
W longitude and 1–16 S latitude. Its
characteristic high precipitation levels make it a good study
region, representing a worst-case scenario with frequent rain
and distortion events. The mean, second-highest value, MMA,
and hybrid composite images of this region were created for all
vertically polarized SSM/I channels. Examples are presented
in Figs. 6 and 7. In the interest of space, only the 37- and
85-V images are shown here. Due to smaller 3-dB antenna
footprints, the higher frequency images exhibit better effective
resolution.
Authorized licensed use limited to: Brigham Young University. Downloaded on February 2, 2009 at 16:49 from IEEE Xplore. Restrictions apply.