HAL Id: hal-00650907
https://hal.inria.fr/hal-00650907
Submitted on 6 Jan 2014
HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entic research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.
L’archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diusion de documents
scientiques de niveau recherche, publiés ou non,
émanant des établissements d’enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.
Uncertainty in a chemistry-transport model due to
physical parameterizations and numerical
approximations: An ensemble approach applied to ozone
modeling
Vivien Mallet, Bruno Sportisse
To cite this version:
Vivien Mallet, Bruno Sportisse. Uncertainty in a chemistry-transport model due to physical parame-
terizations and numerical approximations: An ensemble approach applied to ozone modeling. Journal
of Geophysical Research, American Geophysical Union, 2006, 111 (D01302), �10.1029/2005JD006149�.
�hal-00650907�
Uncertainty in a chemistry-transport model due to
physical parameterizations and numerical
approximations: An ensemble approach applied to
ozone modeling
Vivien Mallet and Bruno Sportisse
Centre d’Enseignement et de Recherche en Environnement Atmosphe´rique, E
´
cole Nationale des Ponts et Chausse´es –
E
´
lectricite´ de France Recherche et De´veloppement, Champs-sur-Marne, France
CLIME, Institut National de Recherche en Informatique et en Automatique – E
´
cole Nationale des Ponts et Chausse´es,
Champs-sur-Marne, France
Received 29 April 2005; revised 19 September 2005; accepted 20 October 2005; published 14 January 2006.
[1] This paper estimates the uncertainty in the outputs of a chemistry-transport model due
to physical parameterizations and numerical approximations. An ensemble of 20
simulations is generated from a reference simulation in which one key parameterization
(chemical mechanism, dry deposition parameterization, turbulent closure, etc.) or one
numerical approximation (grid size, splitting method, etc.) is changed at a time.
Intercomparisons of the simulations and comparisons with observations allow us to assess
the impact of each parameterization and numerical approximation and the robustness of the
model. An ensemble of 16 simulations is also generated with multiple changes in the
reference simulation in order to estimate the overall uncertainty. The case study is a four-
month simulation of ozone concentrations over Europe in 2001 performed using the
modeling system Polyphemus. It is shown that there is a high uncertainty due to the physical
parameterizations (notably the turbulence closure and the chemical mechanism). The low
robustness suggests that ensemble approaches are necessary in most applications.
Citation: Mallet, V., and B. Sportisse (2006), Uncertainty in a chemistry-transport model due to physical parameterizations and
numerical approximations: An ensemble approach applied to ozone modeling, J. Geophys. Res., 111, D01302,
doi:10.1029/2005JD006149.
1. Introduction
[2] Chemistry-transport models are now widely used in
air-quality applications ranging from impact studies to daily
forecasts. To date, they perform satisfactory simulations,
both in basic cases such as passive tracer tracking [e.g.,
Nodop, 1997] and in complex cases involving photochem-
ical mechanisms. The reliability of the models is partially
assessed through comparisons with measurements and nu-
merous statistical measures (as those defined by the Envi-
ronmental Protection Agency (EPA) [1991]). These
comparisons are performed with intensive observation peri-
ods from specific campaigns or with daily measurements
from regular monitoring sites. A large set of comprehensive
and reliable 3D Eulerian chemistry-transport models has
been ‘‘validated’’ this way, such as Chimere [Schmidt et al.,
2001], Community Multiscale Air Quality (CMAQ) [Byun
and Ching, 1999], Danish Eulerian Hemispheric Model
(DEHM) [Christensen, 1997], European Monitoring and
Eval uation Programme (EMEP) [Simpson et al., 2003],
European Air Pollution Dispersion (Eurad) [Hass, 1991],
Long Term Ozone Simulation (Lotos) [Builtjes, 1992], and
Polair3D [Boutahar et al., 2004].
[
3] These models have usually been ‘‘tuned’’ in order to
deli ver satisfactory model-to-observation statistics. Also
while the ‘‘validations’’ give the error of the simulations,
they do not give information on the uncertainty associated
with these simulations. The origin of the uncertainty is
threefold: the underlying physical parameterizations (bio-
genic emissions, deposition velocities, turbulent closure,
chemical mechanism, etc.), the input data (land use data,
emission inventories, raw meteorological fields, chemical
data, etc.) and the numerical approximations (mesh sizes,
time s tep and number of chemical species). The best
characterization of the uncertainty would be the probability
density functions of the simulation errors. Computing a
probability density function (PDF) for given model outputs
(such as forecast error statistics) is in practice a difficult
task primarily because of the computational costs.
[
4] There are specific techniques to assess uncertainties.
The first-order derivatives of model outputs with respect to
model inputs can give ‘‘local’’ estimates of uncertainties
[e.g., Schmidt, 2002]. Monte Carlo simulations based on
different values for given input parameters or fields can
provide an approximation to the probability density functions
if the number of simulations is large enough [Hanna et al.,
2001]. An alternative approach, which is now widely used in
meteorology [Toth and Kalnay, 1993; Houtemaker et al.,
JOURNAL OF GEOPHYSICAL RESEARCH, VOL. 111, D01302, doi:10.1029/2005JD006149, 2006
Copyright 2006 by the American Geophysical Union.
0148-0227/06/2005JD006149$09.00
D01302 1of15
1996; Buizza et al., 1999] and which is a promising method in
air quality modeling (e.g., Delle Monache and Stull [2003]
for photochemistry or Galmarini et al. [2004] for radio-
nuclides), is the so-called ensemble approach based on a set
of models supposed to account for the range of uncertainties.
[
5] This paper uses an ensemble approach to provide
estimates of the uncertainty in photochemical forecasts due
to the parameterizations and some data associat ed with
them. It also deals with numerical issues such as mesh size.
[
6] The study is performed with a four-month European-
scale simulation, from May to August 2001. A comparison
between the reference simulation and a similar simulation
but for one change in a parameterization enables us to
estimate the impact of this parameterization. For each
modified parameterization, the reliability of the simulation
is checked with comparisons to measurements, which
allows us to assess the robustness of the whole modeling
system. The same experiment is finally performed with a set
of simulations in which several parameterizations may be
changed (at the same time, in the same simulation). This
allows us to study the robustness of the system with respect
to cumulated uncertainties.
[
7] This paper is organized as follows. Section 2 briefly
summarizes the relevant methods to estimate uncer tainties,
details the specific aims of this paper and describes the
methodology. Section 3 details the model, the reference
simulation and the involved parameterizations. In the last
section, the results are analyzed with intercomparisons of
the simulations and comparisons to observations.
2. Methodology
2.1. Definitions
[8] We define the following: (1) The error is the discrep-
ancy between model outputs and field observations. (2) The
uncertainty is the range of values in which the model
outputs may lie with a high degree of confidence. In this
paper, we only deal with a priori uncertainties, i.e., unce r-
tainties estimated without taking into account observations.
(3) Hereafter we refer to the variability of an ensemble as its
spread. The spread is a measure of the uncertainty and it can
be quantified by a standard deviation. (4) Herein the
variability solely refers to the spatial or/and temporal
variabilities of a concentration field. For the sake of clarity,
the variability of an ensemble is called a spread.
2.2. Motivation
[
9] Assessing the uncertainties in model outputs is a field
of growing interest in environmental forecasting, especially
in meteorology. In meteorology, the dynamics of models
have a ‘‘chaotic’’ behavior. The uncertainties in initial
conditions have therefore a strong impact and the issue is
to propagate these uncertainties through ‘‘ensemble fore-
casts’’ [Toth and Kalnay, 1993; Houtemaker et al., 1996;
Buizza et al., 1999]. In air quality applications, there is not
such a strong dependence on initial conditions. The impact
of uncertainties in the input data (e.g., emissions, meteoro-
logical fields), in the parameterizations (e.g., deposition
velocities, turbulence closure) and in the numerical algo-
rithms is much stronger.
[
10] The actual errors of a model, given by comparisons
to observational data, may be low with high uncertainties in
the results. A model may be tuned to fit the observations
(and all models are improved this way), which leads to low
errors. Nevertheless, if this model is used with different
parameterizations (assumed to be valid physical parameter-
izations), other data or alternative numerical schemes, then
it could lead to very different results, including those far
from the measurem ents, with the magnitude of sprea d
depending on the actual uncertainty. This is, of course, a
strong limitation of the models, and the uncertainty has to
be estimated in order to assess the ‘‘robustness’’ of the
models. One may refer to Russell and Dennis [2000] for an
overview of the strengths and limitations of photochemical
models.
[
11] It is impossible to compute the error in all meteoro-
logical conditions, at every point in a given simulation
domain (even at ground level), for all chemical species,
and at every time. In the absence of observations, an
estimation of the uncertainty is essentially the only means
to assess the quality of the results. In an operational context,
the models may be used for risk assessment. The reliability
of the results is then a crucial issue and, if available, the full
PDFs associated with these results would be highly valu-
able. For instance, in prospective or screening studies (e.g.,
impact studies related to different emission scenarios), the
models may be used with uncommon i nput data (e.g.,
strongly corrected emissions) and without any available
observations with which to tune the models. From the
research point of view, an estimate of the uncertainty is
necessary for other communities to assess the feasibility and
the relevance of given applications. For instance, the effect
of pollution on health may or may not be effectively
estimated, depending on the accuracy of the underlying
air-quality models. For each model, the development is also
oriented to improve the description in the parameterizations
responsible for the main uncertainty.
2.3. A Review of Existing Methods
[
12] There are several methods to estimate the uncertainty
and to identify its sources. As for the uncertainty due to the
input data, one can compute first-order derivatives of the
model outputs with respect to the model inputs [e.g.,
Schmidt, 2002]. This provides ‘‘local’’ sensitivities from
which the uncertainty in the outputs can be derived, taking
into account the uncertainty in the input data.
[
13] Ideally, one would want to compute the full PDF
associated with the results. This would mean solving the
Fokker-Planck equation (the equation satisfied by the output
PDF [Gardiner, 1996]) which is unfeasible. Instead, the
Monte Carlo methods can generate approximations of the
PDF. The idea is to generate a set of N input fields that
roughly describe the PDF associated with the input data.
The model is then run N times, which provides an approx-
imation of the output PDF. These methods may be well
suited but they are restricted to the uncertainty due to input
data or parameters in parameterizations, that is, due to
continuous variables. A related method, which could be
viewed as a Monte Carlo method too, is the use of a set of N
input fields generated by another model. In practice, the
ensemble forecasts from the meteorological centers may be
used as input to the air quality models. This leads to
promising applications but it is restricted to the meteoro-
logical fields [Warner et al., 2002].
D01302 MALLET AND SPORTISSE: CHEMISTRY-TRANSPORT MODEL UNCERTAINTY
2of15
D01302
[14] Another method is the use of different air quality
models. This technique has already been used but with a
fairly low number of models (e.g., four models given by
Delle Monache and Stull [2003]). It is hard to assemble
enough models to claim a reliable estimate of the uncer-
tainty. Moreover, intercomparisons are difficult because the
models may not be operated under the same conditions
(e.g., with the same meteorological fields). Note that this
technique involves the uncertainties of several models and
is not suited to assess the uncertainty of a given model.
Moreover, the models have usually been tuned in compar-
isons to measured data; hence they do not embrace the
whole uncertainty in the physics and the chemistry.
[
15] The method applied in this pap er main ly takes
advantage of the multiple parameterizations that should be
available in a well designed modeling system [Malletetal.,
2005]. The model is run in many conf igura tions with
respect to the available state-of-the-art parameterizations,
but also with respect to changes in the parameters and the
base input data needed for these parameterizations. The
impact of the numerical approximations is studied as well.
This method allows fair comparisons since the framework is
exactly the same for all simulations. It gives an accurate
view of the uncertainty due to the parameterizations of a
given model. Notice that the method introduces discrete
changes, which is the only means to assess the impact of the
parameterizations. There is no continuous transition be-
tween two parameterizations or between their base input
data sets. Details about the method are provided below.
2.4. Multiconfiguration Approach
[
16] The air quality system with which the experiments
have been performed relies on many parameterizations (see
section 3.1). There are often several valid parameterizations
to compute the same field. Furthermore most parameter-
izations depend on input data sets (including scalar param-
eters). For instance, the deposition velocities depend on the
land use coverage which may be given by U.S. Geological
Survey (USGS) data or by Global Land Cover Facility
(GLCF) data (see below). The alternatives between the
parameterizations themselves and their input data sets
introduce a finite number of choices. Hence the method
deals with discrete dependencies.
[
17] The impact of numerical options are also assessed
through discrete changes, e.g., by changing a numerical
scheme. Nonetheless a few values that belong to a contin-
uous interval are studied as well. They are modified as if
they were discrete variables, i.e., only a few values are
allowed for them. For example, the time step is a continuous
variable but it can be restricted to a set of three values (a
reference time step, a small one and a large one).
[
18] For the sake of clarity, the changes in the input data
to the parameterizations will be viewed as changes in the
parameterizations themselves. Since the numerical issues
are treated in the same way as the parameterizations (they
are associated with a finite number of choices), they are also
viewed as parameterizations hereafter.
[
19] Assume that the model is written in the form
y ¼ fp
1
; p
2
; ...; p
N
ðÞ¼fpðÞ ð1Þ
Every input parameter p
i
2 {0, ..., n
i
1} is associated
with a given parameterization that has n
i
possible values. f is
the model itself. The output y may be the pollutant
concentrations, deposition fields, evaluation statistics, etc.
Notice that f is already a discretized model.
[
20] The reference simulation is associated with a refer-
ence vector assumed to be zero: p
ref
= 0. The idea is to
estimate the uncertainty and the impact of every parameter-
ization by changing one paramete rization at a time, i.e.,
computing all f ( p) where p
i
= 0 for all i except for one
component. There are
P
N
i¼1
(n
i
1) such simulations. This
is only a small subset of the
N
i¼1
n
i
possible combinations,
but the computational cost makes it impossible to run all
simulations.
[
21] This method allows us to estimate the impact of each
parameterization. The impact is estimated with the resulting
changes in the output concentrations. It is analyzed with the
concentration distributions and their spatial and temporal
variabilities. In addition, for each change, an evaluation of
the output can be performed. It shows whether the modified
parameterization leads to an improved agreement with the
measurements and therefore maybe to a better description of
the physics. The fact that not all combinations (p
i
)
i
are
available restricts the study: it is hard to decide whether a
parameterization should be discarded because its drawbacks
may be canceled by changes in other parameterizations.
There are still useful conclusions to draw: for instance, it
may be shown that a given parameterization limits the
variability in the results.
[
22] Furthermore, the results are enhanced by combined
changes, but only with a few selected parameterizations to
reduce the computational cost of the study. Four parameter-
izations are selected mainly because of their significant
impact (even if other parameterizations have a similar
importance). The model is then put in the form y = f (
~
p)
where the vector
~
p has four components. Each component
can take two values (0 or 1); therefore there are 16 possible
combinations. This provides a rough estimate of the overall
uncertainty.
3. Experiment Setup
3.1. Modeling System
[23] This study is based on the modeling system Poly-
phemus (available under the GNU General Public License
at http://ww w.enpc.fr/cerea/polyphemus/). This system is
divided into four parts: (1) The databases incorporate the
data needed in the parameterizations (one may also include
the meteorological fields here). (2) The libraries provide
(a) facilities to manage the multidimensional data involved
in atmospheric chemistry, (b) useful functions associated
with the physical and chemical fields (e.g., coordinate
transformations) and (c) the parameterizations. (3) A set
of programs make the calls to the libraries to generate
the input data needed by the chemistry-transport model.
Their flexibility is made possible by the input configuration
files that they read. (4) The chemistry-transport model
is responsible for the time integration of the chemistry-
transport equation. It therefore computes the output
concentrations.
[
24] The databases contain the raw data: the land use
coverage, the anthropogenic emission inventories, chemical
constants, etc. The meteorological fields may also be
included even if they strongly depend on the application.
D01302 MALLET AND SPORTISSE: CHEMISTRY-TRANSPORT MODEL UNCERTAINTY
3of15
D01302
[25] The libraries play a major role in this study since
they provide the basis of the flexibility of the parameter-
izations. They first provide the data structures and functions
needed for data processing. They then provide a set of
parameterizations. Most of the changes to the simulations
are made with a different call to the libraries, specifically to
the library dedicated to physical parameterizations, the C++
library AtmoData [Mallet and Sportisse, 2005].
[
26] The programs of Polyphemus make calls to the
libraries in order to process the raw data. They format the
raw data for the chemistry-transport model, but the primary
function of the programs is to use the parameterizations
from the library AtmoData to compute the needed fields.
These programs read configuration files in which many
options are specified, including which parameterizations are
to be used and wi th which input data and parameters.
Roughly speaking, there exists a set of configuration files
for every vector p (vector defined in section 2.4). This study
therefore relies heavily on the flexibility characteristic of the
programs.
[
27] Finally, the Eulerian chemistry-t ransport model
Polair3D computes the output concentrations through the
numerical integration of the transport-chemistry equation.
With respect to this study, a strong advantage of Polair3D is
its ability to deal with multiple chemical mechanisms.
Details about Polair3D are given by Boutahar et al. [2004].
[
28] Further details about the architecture of the whole
system are given by Mallet et al. [2005]. A complete
description is not relevant here because not all features of
the system are used in this study. The system is able to
handle many applications (many chemical mechanisms,
data assimilation, Monte Carlo simulations, etc.) and its
flexibility enables the multiple experiments presented in this
paper. The next subsection describes the base application.
3.2. Reference Simulation
[
29] The impact of the parameterizations is evaluated by
the changes they introduce with respect to the reference
simulation. The reference simulation takes place at Europe-
an scale during summer 2001 (22 April 2001 to 31 August
2001). A validation, over the same domain and the same
period, similar to the reference simulation, is given by
Mallet and Sportisse [2004].
[
30] The do main is [40 .25N, 10.25W] [56.75N,
22.25E] and is shown in Figure 1. The first layer is located
between 0 m and 50 m; the concentrations are thus
computed at 25 m. The thickness of the other layers is
about 600 m with the top of the last layer at 3000 m. RACM
is the photochemical mechanism used in this simulation
[Stockwell et al., 1997]. Since the best results are obtained
for ozone and the number of ozone measurements is
significantly higher than for other species, this study focuses
on ozone. We are notably concerned with the ozone peaks
since they are often of high interest in forecasts (because of
the regulations that mostly limit the peaks).
[
31] Here is a review of the main components of the
reference simulat ion: (1) meteorological data (the b est
ECMWF data available for the period (i.e., 0.36 0.36,
the TL511 spectral resolution in the horizontal, 60 levels,
time step of 3 hours, 12 hours forecast cycles starting from
analyzed fields)); (2) land use coverage (USGS finest land
cover map (24 categories, 1 km Lambert)); (3) emissions
(the Co-operative Programme for Monitoring and Evalua-
tion of the Long-range Transmission of Air Pollutants
in Europe (EMEP) inventory, converted according to
Middleton et al. [1990]); (4) biogenic emissions (computed
as advocated by Simpson et al. [1999]); (5) deposition
velocities (the revised parameterization proposed by Zhang
et al. [2003]); (6) vertical diffusion (within the boundary
layer, the Troen and Mahrt parameterization as described by
Troen and Mahrt [1986], with the boundary layer height
provided by the ECMWF; above the boundary layer, the
Louis parameterization [Louis, 1979]); (7) boundary con-
dit ions (output of the global chem istry-transport model
Mozart 2 [Horowitz et al., 2003] run over a typical year);
and (8) numerical schemes (a first-order operator splitting,
the sequence being advection–diffusion–chemistry; a direct
space-time third-order advection scheme with a Koren flux
limiter [Verwer et al., 1998]; a second-order Rosenbrock
method for diffusion and chemistry).
[
32] The performance of the reference simulation has
been evaluated through a comparison of the forecasted
ozone peaks with th e observations from 242 stations
distributed over Europe (in a network with mixed
stations: urban, periurban and rural stations). With the
first five days excluded (because of the rough initial
conditions), the root mean square (with all observ ations
put together) is 23.5 mgm
3
, the correlation is 71.4% and the
bias 4.5 mgm
3
(the mean of observed values being
94.7 mgm
3
): the statistical measures are defined in Appen-
dix A. The results therefore show a reasonable agreement
with observations [Hass et al., 1997; Schmidt et al., 2001].
3.3. Parameterizations
[
33] The modified parameterizations were chosen accord-
ing to the relevance and the availability of alternative
parameterizations. Only stat e-of-the-art parameterizations
or, at least, widely used parameterizations are involved.
The list of the parameterizations (and the data associated
with them) used in this study is shown in Table 1.
[
34] The changes first include prominent processes such
as the chemistry (RADM 2). Several chemical mechanisms
Figure 1. Domain [40.25N, 10.25W] [56.75N,
22.25E] of the reference simulation.
D01302 MALLET AND SPORTISSE: CHEMISTRY-TRANSPORT MODEL UNCERTAINTY
4of15
D01302
![Figure 4. Relative frequency distribution f([O3]) of the ozone concentrations (mg m 3) of the reference simulation and the functions f([O3]) ± sf ([O3]), where sf ([O3]) is the standard deviation of f([O3]) computed from all simulations.](/figures/figure-4-relative-frequency-distribution-f-o3-of-the-ozone-3nxpxq7n.webp)
![Figure 9. Relative standard deviation of the ensemble for concentrations in [40 mg m 3, 130 mg m 3]. The average standard deviation is 17%. The uncertainty is notably high along the coasts.](/figures/figure-9-relative-standard-deviation-of-the-ensemble-for-2bbwcep6.webp)
