HAL Id: hal-00271141
https://hal.archives-ouvertes.fr/hal-00271141
Submitted on 21 Jan 2010
HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entic research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.
L’archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diusion de documents
scientiques de niveau recherche, publiés ou non,
émanant des établissements d’enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.
A review of image denoising algorithms, with a new one
Antoni Buades, Bartomeu Coll, Jean-Michel Morel
To cite this version:
Antoni Buades, Bartomeu Coll, Jean-Michel Morel. A review of image denoising algorithms, with
a new one. Multiscale Modeling and Simulation: A SIAM Interdisciplinary Journal, Society for
Industrial and Applied Mathematics, 2005, 4 (2), pp.490-530. �hal-00271141�
A REVIEW OF IMAGE DENOISING ALGORITHMS, WITH A NEW
ONE.
A. BUADES
† ‡
, B. COLL
†
, AND J.M. MOREL
‡
Abstract. The search for efficient image denoising methods still is a valid challenge, at the
crossing of functional analysis and statistics. In spite of the sophistication of the recently proposed
methods, most algorithms have not yet attained a desirable level of applicability. All show an out-
standing performance when the image model corresponds to the algorithm assumptions, but fail in
general and create artifacts or remove image fine structures. The main focus of this paper is, first, to
define a general mathematical and experimental methodology to compare and classify classical image
denoising algorithms, second, to propose an algorithm (Non Local Means) addressing the preservation
of structure in a digital image. The mathematical analysis is based on the analysis of the “method
noise”, defined as the difference between a digital image and its denoised version. The NL-means
algorithm is proven to be asymptotically optimal under a generic statistical image model. The de-
noising performance of all considered methods are compared in four ways; mathematical: asymptotic
order of magnitude of the method noise under regularity assumptions; perceptual-mathematical: the
algorithms artifacts and their explanation as a violation of the image model; quantitative experi-
mental: by tables of L
2
distances of the denoised version to the original image. The most powerful
evaluation method seems, however, to be the visualization of the method noise on natural images.
The more this method noise looks like a real white noise, the better the method.
Key words. Image restoration, non parametric estimation, PDE smoothing filters, adaptive
filters, frequency domain filters
AMS subject classifications. 62H35
1. Introduction.
1.1. Digital images and noise. The need for efficient image restoration meth-
ods has grown with the massive production of digital images and movies of all kinds,
often taken in poor conditions. No matter how good cameras are, an image improve-
ment is always desirable to extend their range of action.
A digital image is generally encoded as a matrix of grey level or color values. In
the case of a movie, this matrix has three dimensions, the third one corresponding
to time. Each pair (i, u(i)) where u(i) is the value at i is called pixel, for “picture
element”. In the case of grey level images, i is a point on a 2D grid and u(i) is a
real value. In the case of classical color images, u(i) is a triplet of values for the red,
green and blue components. All of what we shall say applies identically to movies,
3D images and color or multispectral images. For a sake of simplicity in notation
and display of experiments, we shall here be contented with rectangular 2D grey-level
images.
The two main limitations in image accuracy are categorized as blur and noise.
Blur is intrinsic to image acquisition systems, as digital images have a finite number of
samples and must satisfy the Shannon-Nyquist sampling conditions [32]. The second
main image perturbation is noise.
†
Universitat de les Illes Balears, Anselm Turmeda, Ctra. Valldemossa Km. 7.5, 07122 Palma de
Mallorca, Spain. Author financed by the Ministerio de Ciencia y Tecnologia under grant TIC2002-
02172. During this work, the first author had a fellowship of the Govern de les Illes Balears for the
realization of his PhD.
‡
Centre de Mathmatiques et Leurs Applications. ENS Cachan 61, Av du Prsident Wilson 94235
Cachan, France. Author financed by the Centre National d’Etudes Spatiales (CNES), the Office
of Naval Research under grant N00014-97-1-0839, the Direction Gnrale des Armements (DGA), the
Ministre de la Recherche et de la Technologie.
1
2 A. BUADES, B. COLL AND J.M MOREL
Each one of the pixel values u(i) is the result of a light intensity measurement,
usually made by a CCD matrix coupled with a light focusing system. Each captor
of the CCD is roughly a square in which the number of incoming photons is being
counted for a fixed period corresponding to the obturation time. When the light
source is constant, the number of photons received by each pixel fluctuates around
its average in accordance with the central limit theorem. In other terms one can
expect fluctuations of order
√
n for n incoming photons. In addition, each captor, if
not adequately cooled, receives heat spurious photons. The resulting perturbation is
usually called “obscurity noise”. In a first rough approximation one can write
v(i) = u(i) + n(i),
where i ∈ I, v(i) is the observed value, u(i) would be the “true” value at pixel i,
namely the one which would be observed by averaging the photon counting on a long
period of time, and n(i) is the noise perturbation. As indicated, the amount of noise
is signal-dependent, that is n(i) is larger when u(i) is larger. In noise models, the
normalized values of n(i) and n(j) at different pixels are assumed to be independent
random variables and one talks about “white noise”.
1.2. Signal and noise ratios. A good quality photograph (for visual inspec-
tion) has about 256 grey level values, where 0 represents black and 255 represents
white. Measuring the amount of noise by its standard deviation, σ(n), one can define
the signal noise ratio (SNR) as
SNR =
σ(u)
σ(n)
,
where σ(u) denotes the empirical standard deviation of u,
σ(u) =
Ã
1
|I|
X
i∈I
(u(i) −
u)
2
!
1
2
and
u =
1
|I|
P
i∈I
u(i) is the average grey level value. The standard deviation of the
noise can also be obtained as an empirical measurement or formally computed when
the noise model and parameters are known. A good quality image has a standard
deviation of about 60.
The best way to test the effect of noise on a standard digital image is to add
a gaussian white noise, in which case n(i) are i.i.d. gaussian real variables. When
σ(n) = 3, no visible alteration is usually observed. Thus, a
60
3
≃ 20 signal to noise
ratio is nearly invisible. Surprisingly enough, one can add white noise up to a
2
1
ratio and still see everything in a picture ! This fact is illustrated in Figure 1.1
and constitutes a major enigma of human vision. It justifies the many attempts to
define convincing denoising algorithms. As we shall see, the results have been rather
deceptive. Denoising algorithms see no difference between small details and noise, and
therefore remove them. In many cases, they create new distortions and the researchers
are so much used to them as to have created a taxonomy of denoising artifacts:
“ringing”, “blur”, “staircase effect”, “checkerboard effect”, “wavelet outliers”, etc.
This fact is not quite a surprise. Indeed, to the best of our knowledge, all denoising
algorithms are based on
• a noise model
• a generic image smoothness model, local or global.
A REVIEW OF IMAGE DENOISING ALGORIHTMS WITH A NEW ONE 3
Fig. 1.1. A digital image with standard deviation 55, the same with noise added (standard
deviation 3), the signal noise ratio being therefore equal to 18, and the same with signal noise ratio
slightly larger than 2. In this second image, no alteration is visible. In the third, a conspicuous
noise with standard deviation 25 has been added but, surprisingly enough, all details of the original
image still are visible.
In experimental settings, the noise model is perfectly precise. So the weak point of the
algorithms is the inadequacy of the image model. All of the methods assume that the
noise is oscillatory, and that the image is smooth, or piecewise smooth. So they try
to separate the smooth or patchy part (the image) from the oscillatory one. Actually,
many fine structures in images are as oscillatory as noise is; conversely, white noise
has low frequencies and therefore smooth components. Thus a separation method
based on smoothness arguments only is hazardous.
1.3. The “method noise”. All denoising methods depend on a filtering pa-
rameter h. This parameter measures the degree of filtering applied to the image. For
most methods, the parameter h depends on an estimation of the noise variance σ
2
.
One can define the result of a denoising method D
h
as a decomposition of any image
v as
v = D
h
v + n(D
h
, v),(1.1)
where
1. D
h
v is more smooth than v
2. n(D
h
, v) is the noise guessed by the method.
Now, it is not enough to smooth v to ensure that n(D
h
, v) will look like a noise.
The more recent methods are actually not contented with a smoothing, but try to
recover lost information in n(D
h
, v) [19], [25]. So the focus is on n(D
h
, v).
Definition 1.1 (Method noise). Let u be a (not necessarily noisy) image and
D
h
a denoising operator depending on h. Then we define the method noise of u as
the image difference
n(D
h
, u) = u − D
h
(u).(1.2)
This method noise should be as similar to a white noise as possible. In addition,
since we would like the original image u not to be altered by denoising methods, the
method noise should be as small as possible for the functions with the right regularity.
According to the preceding discussion, four criteria can and will be taken into
account in the comparison of denoising methods:
• a display of typical artifacts in denoised images.
4 A. BUADES, B. COLL AND J.M MOREL
• a formal computation of the method noise on smooth images, evaluating how
small it is in accordance with image local smoothness.
• a comparative display of the method noise of each method on real images
with σ = 2.5. We mentioned that a noise standard deviation smaller than 3 is
subliminal and it is expected that most digitization methods allow themselves
this kind of noise.
• a classical comparison receipt based on noise simulation : it consists of taking
a good quality image, add gaussian white noise with known σ and then com-
pute the best image recovered from the noisy one by each method. A table
of L
2
distances from the restored to the original can be established. The L
2
distance does not provide a good quality assessment. However, it reflects well
the relative performances of algorithms.
On top of this, in two cases, a proof of asymptotic recovery of the image can be
obtained by statistical arguments.
1.4. Which methods to compare ?. We had to make a selection of the de-
noising methods we wished to compare. Here a difficulty arises, as most original
methods have caused an abundant literature proposing many improvements. So we
tried to get the best available version, but keeping the simple and genuine character
of the original method : no hybrid method. So we shall analyze :
1. the Gaussian smoothing model (Gabor [16]), where the smoothness of u is
measured by the Dirichlet integral
R
|Du|
2
;
2. the anisotropic filtering model (Perona-Malik [28], Alvarez et al. [1]);
3. the Rudin-Osher-Fatemi [31] total variation model and two recently proposed
iterated total variation refinements [36, 25];
4. the Yaroslavsky ([42], [40]) neighborhood filters and an elegant variant, the
SUSAN filter (Smith and Brady) [34];
5. the Wiener local empirical filter as implemented by Yaroslavsky [40];
6. the translation invariant wavelet thresholding [8], a simple and performing
variant of the wavelet thresholding [10];
7. DUDE, the discrete universal denoiser [24] and the UINTA, Unsupervised
Information-Theoretic, Adaptive Filtering [3], two very recent new approaches;
8. the non local means (NL-means) algorithm, which we introduce here.
This last algorithm is given by a simple closed formula. Let u be defined in a bounded
domain Ω ⊂ R
2
, then
NL(u)(x) =
1
C(x)
Z
e
−
(G
a
∗|u(x+.)−u(y+.)|
2
)(0)
h
2
u(y) dy,
where x ∈ Ω, G
a
is a Gaussian kernel of standard deviation a, h acts as a filtering
parameter and C(x) =
R
e
−
(G
a
∗|u(x+.)−u(z+.)|
2
)(0)
h
2
dz is the normalizing factor. In order
to make clear the previous definition, we recall that
(G
a
∗ |u(x + .) − u(y + .)|
2
)(0) =
Z
R
2
G
a
(t)|u(x + t) −u(y + t)|
2
dt.
This amounts to say that NL(u)(x), the denoised value at x, is a mean of the values
of all pixels whose gaussian neighborhood looks like the neighborhood of x.
1.5. What is left. We do not draw into comparison the hybrid methods, in
particular the total variation + wavelets ([7], [11], [17]). Such methods are significant
improvements of the simple methods but are impossible to draw into a benchmark :