scispace - formally typeset
Search or ask a question
Journal ArticleDOI

Toward a reliable decomposition of predictive uncertainty in hydrological modeling: Characterizing rainfall errors using conditional simulation

01 Nov 2011-Water Resources Research (John Wiley & Sons, Ltd)-Vol. 47, Iss: 11, pp 1-21
TL;DR: It is shown that independently derived data quality estimates are needed to decompose the total uncertainty in the runoff predictions into the individual contributions of rainfall, runoff, and structural errors.
Abstract: [1] This study explores the decomposition of predictive uncertainty in hydrological modeling into its contributing sources. This is pursued by developing data-based probability models describing uncertainties in rainfall and runoff data and incorporating them into the Bayesian total error analysis methodology (BATEA). A case study based on the Yzeron catchment (France) and the conceptual rainfall-runoff model GR4J is presented. It exploits a calibration period where dense rain gauge data are available to characterize the uncertainty in the catchment average rainfall using geostatistical conditional simulation. The inclusion of information about rainfall and runoff data uncertainties overcomes ill-posedness problems and enables simultaneous estimation of forcing and structural errors as part of the Bayesian inference. This yields more reliable predictions than approaches that ignore or lump different sources of uncertainty in a simplistic way (e.g., standard least squares). It is shown that independently derived data quality estimates are needed to decompose the total uncertainty in the runoff predictions into the individual contributions of rainfall, runoff, and structural errors. In this case study, the total predictive uncertainty appears dominated by structural errors. Although further research is needed to interpret and verify this decomposition, it can provide strategic guidance for investments in environmental data collection and/or modeling improvement. More generally, this study demonstrates the power of the Bayesian paradigm to improve the reliability of environmental modeling using independent estimates of sampling and instrumental data uncertainties.

Summary (5 min read)

1.1. Hydrological Modeling in the Presence of Rainfall and Runoff Errors

  • [2] Data and model errors conspire to make reliable and robust calibration of hydrological models a difficult task.
  • The rain gauges themselves are subject to both systematic and random measurement errors, including mechanical limitations, wind effects, and evaporation losses, all of which are design specific and can vary substantially with rainfall intensity [Molini et al., 2005].
  • Finally, the characterization of structural uncertainty is a particularly challenging task, and the hydrological community has yet to agree on suitable definitions and approaches for handling structural model errors in the 1UR HHLY Hydrology-Hydraulics, Cemagref, Lyon, France.

1.2. Decomposing Predictive Uncertainty

  • The focus of this paper is on the decomposition of the total uncertainty in hydrological predictions into its contributing sources.
  • It helps in more informed research and experimental resource allocation, and, importantly, allow a meaningful a posteriori evaluation of these efforts. [11].
  • In a prediction context, attempts to decompose the total uncertainty into its three main sources have been made using several related methods.
  • The authors consider Bayesian hierarchical approaches [e.g., Huard and Mailhot, 2008; Kuczera et al., 2006], which to date have been implemented in batch estimation form (but can also be formulated recursively).

1.3. Specifying Data and Structural Error Models

  • Output, and structural errors is well known, developing quantitative error models is a considerable challenge in hydrological applications.
  • Similarly, Salamon and Feyen [2010] used literature values for runoff errors ( 12.5% standard error for large runoff) and rule-of-thumb values for rainfall and structural errors ( 15% standard error). [17].
  • These studies did not attempt to fully decompose predictive uncertainty.
  • More recently, Renard et al. [2010] and Kuczera et al. [2010b] quantitatively demonstrated the difficulties of simultaneously identifying rainfall and structural errors from rainfall-runoff data when only vague estimates of data uncertainty are known prior to the hydrological model calibration.
  • This result confirms the earlier discussions by Beven [2005, 2006] of potential interactions between multiple sources of error.

1.4. Study Aims

  • This constitutes a major contribution of this paper since previous attempts at isolating the contribution of structural errors to predictive uncertainty [Huard and Mailhot, 2008; Salamon and Feyen, 2010] were based on assuming known parameters of the structural error model. [22].
  • The Bayesian foundation of BATEA, in particular, its ability to exploit quantitative (though potentially vague) probabilistic insights into individual sources of error, makes it well suited for using independent knowledge to improve parameter inference and predictions and to quantify individual contributions to predictive uncertainties.
  • The development of realistic error models for rainfall and runoff errors is of general interest for any method aiming at decomposing the predictive uncertainty into its three main contributive sources. [23].
  • This work is innovative in several aspects.

2.1. General Setup: Data and Model

  • Nt denote the true rainfall and true runoff time series of length Nt, respectively.
  • Hydrological models are usually also forced with potential evapotranspiration (PET).
  • Sensitivity to PET random errors is minor, and the impact of PET systematic errors remains much smaller than that of rainfall errors [e.g., Oudin et al., 2006].
  • The authors therefore exclude PET uncertainty from the analysis and notation.

2.3. Structural Errors of Rainfall-Runoff Models

  • Unlike data errors, which can be estimated independently from the hydrological model by analyzing the observational network, no widely accepted approaches exist for characterizing structural uncertainty [e.g., see Beven, 2005, 2006; Doherty and Welter, 2010; Kennedy and O’Hagan, 2001; Kuczera et al., 2006].
  • Standard least squares regression corresponds to assuming n Nðnj0; 2 Þ and assuming that this term also accounts for input and output errors. [31].
  • Since structural error remains the least understood source of uncertainty, scarce guidance exists for specifying anything other than vague priors, whether on exogenous structural error terms or on stochastic parameters.

2.4. Remnant Errors

  • In addition to error models developed for particular error sources, the authors also account for ‘‘remnant’’ errors [Renard et al., 2010; Thyer et al., 2009].
  • These are related to the notions of ‘‘model inadequacy’’ [Kennedy and O’Hagan, 2001] and ‘‘model discrepancy’’ [Goldstein and Rougier, 2009] but are intended to capture not only unaccounted structural errors of the hydrological model but also inevitable imperfections and omissions in the descriptions of data uncertainty. [35].
  • Note that in traditional regression, remnant errors such as (8) represent the lumped effects of all sources of error and correspond to ‘‘residual’’ errors.

2.5. Posterior Distribution

  • The ‘‘rainfall likelihood’’ pð~RjR;HRÞ describes rainfalls errors. [41].
  • The posterior in equation (10) can be explored using Markov chain Monte Carlo (MCMC) sampling.
  • This would be needed, for example, if BATEA were applied recursively as new data arrives. [44].
  • The key scientific (as opposed to computational) challenge in using BATEA or any other Bayesian approach for the decomposition of individual sources of error is to develop accurate and precise probabilistic models for the individual terms in the posterior (10).

2.6. Calibration Schemes

  • The BATEA framework can be used to derive several calibration schemes, differing in the type of error models and the amount of prior knowledge utilized in the inference.
  • The OI-CS scheme is an ‘‘enhanced’’ OI scheme, augmented using an informative prior for the term p(R).
  • The output-input-structural (OIS) scheme explicitly accounts for rainfall and runoff uncertainty and characterizes structural errors using a stochastic RR parameter.
  • Pr io r pð h Þ 5 of 21 it still uses the remnant error (8) to account for the inevitable imperfections of the uncertainty models. [50].

2.7.1. Total Predictive Distributions

  • Let N denote the vector of all inferred quantities and pðNj~DÞ denote the posterior of parameters N given observed data ~D.
  • The PPD pðY j~D; 1Þ, representing the uncertainty contributed by 2, can be defined in a similar manner. [56].

3.1.1. The Yzeron Catchment

  • The case study is based on the 129 km2 Yzeron catchment in the Rhône-Alpes region of France, near Lyon .
  • The annual average rainfall and runoff are approximately 845 and 150 mm respectively, yielding an annual runoff coefficient of 0.18.
  • The first set, denoted as R3D, comprises three rain gauges in the lower areas of the catchment , with daily totals available for the whole period of study.
  • This study applies the widely used GR4J model [Perrin et al., 2003], which simulates catchment runoff using rainfall and potential evapotranspiration at a daily time step .

3.2.1. Conditional Simulation

  • The uncertainty of areal rainfall estimates is generally dominated by sampling errors, i.e., errors due to the incomplete description of the rainfall spatial field using rain gauges [Moulin et al., 2009; Severino and Alpuim, 2005].
  • This allows the generated field to be consistent with the spatiotemporal properties of aggregated rainfall. [69].
  • The discrepancies in small rainfall events have several possible explanations: (1) biases in the R3D areal averages due to insufficient spatial coverage and/or (2) biases in the CS of small rainfall events.

3.3. Development of the Runoff Error Model

  • Runoff uncertainty was investigated by analyzing the rating curve and related stage-discharge measurements.
  • Figure 6 shows the runoff measurement errors, defined as the difference between the runoff measurements and the runoff predicted by the rating curve (‘‘RCP runoff’’).
  • Equation (17) was fitted to the Yzeron runoff data (with vague priors on a and b) using the WinBugs software [Spiegelhalter et al., 2003].
  • This deficiency arises because of limited gauging data in the high-flow range (a single measurement for flows exceeding 10 mm). [89].

3.4. Representation of Structural Errors

  • The characterization of structural error of the GR4J model is explored using stochastic daily variation of its parameter 1.
  • The authors also investigate a more traditional exogenous treatment of structural errors using the remnant error term (see also section 2.4). [92].
  • Note that 1 controls the maximum storage of the production store .
  • A separate sensitivity analysis indicated that this parameter, when made stochastic, had the largest impact on model predictions.
  • Importantly, the authors examined the inferred stochastic variability of 1 to determine its effect on the storage values and long-term water balance (section 4.4.2).

4.1. Well Posedness of the Calibration Schemes

  • The convergence of MCMC samples reflects the statistical characteristics of the target distribution.
  • In particular, slowly convergent sampling is often indicative of illposed posteriors [Renard et al., 2010].
  • This emphasizes that the computational cost of an inference depends more on its structure than just on its dimensionality.
  • Moreover, the posterior standard deviations of inferred quantities were higher than in the OIS-CS scheme by a factor of about 3 on average, but exceeding 10 for some latent variables.
  • The nonconvergence of the OIS scheme, contrasted with the convergence of the OIS-CS scheme, supports a key conclusion of Renard et al. [2010], namely, that the specification of informative priors for rainfall and runoff uncertainty is a necessary step to ensure well posedness when both forcing and structural errors are modeled hierarchically using latent variables.

4.2. Reliability of Total Predictive Uncertainty (All Schemes)

  • [99] Section 4.2 examines the adequacy of the predictive distribution of runoff.
  • Moreover, the reliability of the total predictive distribution does not prove that all individual error models are correctly specified; it is a necessary but insufficient condition.
  • The OI-CS scheme slightly underestimates the predictive uncertainty, with about 1% of the observations lying outside of the predictive range (p-values of 0 and 1 by convention).
  • The OIS-CS scheme yields a reliable estimation of the predictive uncertainty for all runoff ranges .

4.3. Decomposition of Total Predictive Uncertainty Into Forcing, Response, and Structural Components (OIS-CS Only)

  • [108] Section 4.2 showed that the BATEA methodology yields reliable estimates of predictive uncertainty when prior information on rainfall and runoff errors is available (OIS-CS scheme).
  • Figure 9 shows the TPD and PPD for the forcing, structural, and response errors (see section 2.7 for details).
  • Under the hypotheses made in this case study (including the hydrological model and the data error models), predictive uncertainty in the runoff appears to be dominated by structural errors.
  • AThe posterior means are reported, followed by the corresponding posterior standard deviations.
  • Note that this study uses partial predictive intervals in the decomposition of uncertainty.

4.4.1. Input Errors (OIS-CS Only)

  • Figure 10 shows diagnostic plots to scrutinize the rainfall error model in equation (16).
  • URFs for smaller rainfall events are highly variable, with some multipliers having a significant reduction in uncertainty after calibration.
  • When the store exceeds its capacity, some water is subtracted from the production store.
  • Accounting for data errors (OI-CS, OI, and OISCS schemes) markedly reduces the skewness and excess kurtosis of the standardized residuals.
  • This further discredits the assumption of homoscedastic Gaussian remnant errors and needs to be addressed. [125].

5.1. Quantification of Predictive Uncertainty

  • Section 4.2 indicated that the predictive distribution of runoff was fairly reliable for the OI-CS and OIS-CS scheme.
  • It can be seen that the OI-CS scheme slightly underestimates predictive uncertainty , while the OI scheme yields significantly larger estimated input errors and predictive uncertainty.
  • This suggests that the CS prior constrains the input error estimates and reduces their ability to interact with structural errors and compensate for unaccounted errors. [127].
  • Importantly, the OIS scheme, which omits prior information on the rainfall errors, leads to an ill-posed inference (section 4.1).
  • This is consistent with previous findings that priors on rainfall and runoff uncertainty control the well posedness of Bayesian hierarchical inferences in hydrology [Renard et al., 2010].

5.2. Decomposition of Predictive Uncertainty

  • The empirical results in section 4.3 suggest that decomposing predictive uncertainty into its contributing sources is possible when independent estimates of rainfall and runoff data uncertainty are available and used in the BATEA inference.
  • The reliability of this decomposition can be examined by considering (1) the reliability of the total predictive distribution in combination with (2) the reliability of the individual data and structural components.
  • In applications where longer periods of densely gauged rainfall are available, it could be partitioned between calibration and validation. [130].
  • Direct validation of the estimated structural uncertainty requires highly accurate forcing and response data, so that structural errors can be isolated.

5.3. On the Treatment of Structural Error

  • The treatment of structural error remains a topic of active research (e.g., see the discussion by Beven [2005] and Doherty and Welter [2010]).
  • This study does not aim to compare or improve methods for representing structural errors.
  • Instead, it uses two particular structural error methods as part of a study pursuing error decomposition by exploiting independently derived data error models.
  • The authors view this as a logical first step before structural error characterization is tackled. [133].
  • Which of these approaches, if any, provide an adequate description of structural errors remains an open question.

6. Conclusions

  • The application of the Bayesian framework in a real-data case study confirms earlier findings that prior information on data uncertainty is not merely beneficial but essential for a meaningful and reliable quantification and decomposition of the predictive uncertainty. [147].
  • Conversely, including additional error models improved the reliability of the total uncertainty estimates.
  • The authors stress that this improvement was demonstrated in the validation period and thus is unlikely to be due to potential overfitting. [149].
  • Given the manifest significance of a robust quantitative understanding of data and modeling uncertainties in environmental studies, further development and implementation of instrumental and statistical procedures is needed to estimate the accuracy and precision of environmental data at the data collection and postprocessing stages.

Did you find this useful? Give us your feedback

Figures (12)

Content maybe subject to copyright    Report

PUBLISHED VERSION
Renard, Benjamin; Kavetski, Dmitri; Leblois, E.; Thyer, Mark Andrew; Kuczera, George; Franks, S. W.
Toward a reliable decomposition of predictive uncertainty in hydrological modeling: Characterizing rainfall
errors using conditional simulation
Water Resources Research, 2011; 47:W11516
Copyright 2011 by the American Geophysical Union.
The electronic version of this article is the complete one and can be found online at:
http://onlinelibrary.wiley.com/doi/10.1029/2011WR010643/abstract
http://hdl.handle.net/2440/70957
PERMISSIONS
http://publications.agu.org/author-resource-center/usage-permissions/#repository
Permission to Deposit an Article in an Institutional Repository
Adopted by Council 13 December 2009
AGU allows authors to deposit their journal articles if the version is the final published citable version
of record, the AGU copyright statement is clearly visible on the posting, and the posting is made 6
months after official publication by the AGU.
20
th
May 2013

Toward a reliable decomposition of predictive uncertainty in
hydrological modeling: Characterizing rainfall errors using
conditional simulation
Benjamin Renard,
1
Dmitri Kavetski,
2
Etienne Leblois,
1
Mark Thyer,
3
George Kuczera,
2
and Stewart W. Franks
2
Received 8 March 2011; revised 29 September 2011; accepted 1 October 2011; published 16 November 2011.
[1] This study explores the decomposition of predictive uncertainty in hydrological
modeling into its contributing sources. This is pursued by developing data-based probability
models describing uncertainties in rainfall and runoff data and incorporating them into the
Bayesian total error analysis methodology (BATEA). A case study based on the Yzeron
catchment (France) and the conceptual rainfall-runoff model GR4J is presented. It exploits a
calibration period where dense rain gauge data are available to characterize the uncertainty
in the catchment average rainfall using geostatistical conditional simulation. The inclusion
of information about rainfall and runoff data uncertainties overcomes ill-posedness
problems and enables simultaneous estimation of forcing and structural errors as part of the
Bayesian inference. This yields more reliable predictions than approaches that ignore or
lump different sources of uncertainty in a simplistic way (e.g., standard least squares). It is
shown that independently derived data quality estimates are needed to decompose the total
uncertainty in the runoff predictions into the individual contributions of rainfall, runoff, and
structural errors. In this case study, the total predictive uncertainty appears dominated by
structural errors. Although further research is needed to interpret and verify this
decomposition, it can provide strategic guidance for investments in environmental data
collection and/or modeling improvement. More generally, this study demonstrates the
power of the Bayesian paradigm to improve the reliability of environmental modeling using
independent estimates of sampling and instrumental data uncertainties.
Citation: Renard, B., D. Kavetski, E. Leblois, M. Thyer, G. Kuczera, and S. W. Franks (2011), Toward a reliable decomposition of
predictive uncertainty in hydrological modeling: Characterizing rainfall errors using conditional simulation, Water Resour. Res., 47,
W11516, doi:10.1029/2011WR010643.
1. Introduction
1.1. Hydrological Modeling in the Presence of Rainfall
and Runoff Errors
[
2] Data and model errors conspire to make reliable and
robust calibration of hydrological models a difficult task.
Consequently, a multitude of paradigms for model estima-
tion and prediction have been proposed and used over the
last few decades, ranging from optimization approaches to
probabilistic inference schemes (e.g., see the review by
Moradkhani and Sorooshian [2008]).
[
3] The use of rain gauges to estimate catchment average
precipitation is currently prevalent in hydrological modeling
[Moulin et al., 2009]. A major source of uncertainty is then
the poor representativeness of an often small set of gauges of
the entire areal rain field, which is highly variable in both
space and time [e.g., Severino and Alpuim, 2005; Villarini
et al., 2008]. The rain gauges themselves are subject to both
systematic and random measurement errors, including me-
chanical limitations, wind effects, and evaporation losses, all
of which are design specific and can vary substantially with
rainfall intensity [Molini et al., 2005]. Methods for quantify-
ing rainfall uncertainty include geostatistical approaches such
as kriging [e.g., Goovaerts, 2000; Kuczera and Williams,
1992] and conditional simulation [e.g., Clark and Slater,
2006; Gotzinger and Bardossy, 2008; Onibon et al.,2004;
Vischeletal., 2009] or approaches based on dense rain gauge
networks [e.g., Villarini et al.,2008;Willems, 2001].
[
4] Similarly, runoff data also contain significant observa-
tional errors because of discharge gauging errors, extrapola-
tion of rating curves, unsteady flow conditions, flow regime
hysteresis, and temporal changes in the channel properties.
Several approaches have been proposed to quantify this
uncertainty [e.g., Di Baldassarre and Montanari,2009;
Herschy,1994;Lang et al.,2010;McMillan et al.,2010;
Reitan and Petersen-Overleir,2009].
[
5] Finally, the characterization of structural uncertainty
is a particularly challenging task, and the hydrological
community has yet to agree on suitable definitions and
approaches for handling structural model errors in the
1
UR HHLY Hydrology-Hydraulics, Cemagref, Lyon, France.
2
School of Engineering, University of Newcastle, Callaghan, New
South Wales, Australia.
3
School of Civil, Environmental and Mining Engineering, University of
Adelaide, Adelaide, South Australia, Australia.
Copyright 2011 by the American Geophysical Union.
0043-1397/11/2011WR010643
W11516 1of21
WATER RESOURCES RESEARCH, VOL. 47, W11516, doi:10.1029/2011WR010643, 2011

context of model calibration (e.g., see the conceptualiza-
tions proposed by Beven [2005], Doherty and Welter
[2010], and Kuczera et al. [2006]).
1.2. Decomposing Predictive Uncertainty
[
6] The focus of this paper is on the decomposition of
the total uncertainty in hydrological predictions into its
contributing sources. This is important in several scientific
and operational contexts:
[
7] 1. In operational prediction, separating data and
structural uncertainties is important when data of differing
quality are used in calibration and prediction.
[
8] 2. Separating data and structural uncertainties also
enables a more meaningful model comparison because
structural errors are not obscured by data uncertainty.
[
9] 3. Insights into the relative contributions of data and
model structural errors may be useful when a calibrated
model is transferred to a different catchment (prediction in
ungauged basins). In addition, potential relationships between
catchment characteristics and hydrological model parameters
maybehiddenorbiasedbydataerrors.
[
10] 4. Insights into the relative contributions of individual
sources of error suggest strategic guidance for reducing total
predictive uncertainty. It helps in more informed research
and experimental resource allocation, and, importantly, allow
a meaningful a posteriori evaluation of these efforts.
[
11] Uncertainty decomposition has a considerable his-
tory in the hydrologic forecasting community. For example,
the Bayesian forecasting system (BFS) [Krzysztofowicz,
1999, 2002] distinguishes between two sources of uncertain-
ties in hydrologic forecasts: (1) ‘input uncertainty’ refers
to the uncertainty in forecasting an unknown future rainfall,
and (2) ‘hydrologic uncertainty’ collectively refers to all
other uncertainties, in particular structural errors of the
hydrologic model, parameter estimation errors, input-output
measurement and sampling errors [Krzysztofowicz, 1999].
[
12] This description highlights a major difference
between the uncertainty decomposition in forecasting mode
versus the decomposition in prediction mode. In the for-
mer, input uncertainty is due to forecast errors, while in the
latter, input uncertainty is due to errors in the estimation of
areal rainfall using observations. Note that the word predic-
tion is used here to denote an application where the hydro-
logic model is forced with observed inputs (as opposed to
forecasted inputs).
[
13] This paper focuses on decomposing uncertainty in
the prediction context. This can be viewed as an attempt to
further decompose what is termed ‘hydrologic uncertainty’
in Krzysztofowicz’s [1999] BFS framework. Although Seo
et al. [2006] discussed the potential benefits of such an
additional decomposition, it is usually not viewed as a
major objective because at least for forecast lead times
exceeding the routing time of the catchment, rainfall fore-
cast uncertainty will usually dominate other sources of error
[Krzysztofowicz, 1999]. However, the situation is different
in a prediction context, where no rainfall forecast is
involved. In this case, the relative contributions of rainfall,
runoff, and structural errors to the total predictive uncer-
tainty are unclear and likely case specific.
[
14] In a prediction context, attempts to decompose the
total uncertainty into its three main sources have been
made using several related methods. Multiple studies have
employed recursive data assimilation methods such as
extended and ensemble Kalman filters [Evensen, 1994;
Moradkhani et al., 2005b; Rajaram and Georgakakos,
1989; Reichle et al., 2002; Vrugt et al., 2005] or Bayesian
filtering [Moradkhani et al., 2005a, 2006; Salamon and
Feyen, 2009; Smith et al., 2008; Weerts and El Serafy,
2006]. In this paper, we consider Bayesian hierarchical
approaches [e.g., Huard and Mailhot, 2008; Kuczera et al.,
2006], which to date have been implemented in batch esti-
mation form (but can also be formulated recursively).
While the distinction between recursive versus batch proc-
essing strategies is important from the computational per-
spective, our focus here is on the fundamental issues of the
derivation of informative error models and their incorpora-
tion into the inference framework.
1.3. Specifying Data and Structural Error Models
[
15] Although the importance of adequate descriptions of
input, output, and structural errors is well known, developing
quantitative error models is a considerable challenge in
hydrological applications. In particular, assigning reasonable
values to the variances of rainfall and runoff errors is notori-
ously difficult [e.g., Huard and Mailhot, 2008; Reichle,
2008; Weerts and El Serafy, 2006]. The characterization of
structural errors of hydrological models is also a major
research challenge (e.g., see the discussions by Beven [2005],
Doherty and Welter [2010], and Renard et al. [2010]).
[
16] As a result, it is currently common to use rule-of-
thumb or literature values to fully specify the input, output,
and structural error models and keep their parameters fixed
during the hydrological model calibration. For example,
Huard and Mailhot [2008] used literature values for rainfall
errors and rule-of-thumb values for structural errors (15%
standard error). Similarly, Salamon and Feyen [2010] used
literature values for runoff errors (12.5% standard error
for large runoff) and rule-of-thumb values for rainfall and
structural errors (15% standard error).
[
17] However, recent empirical and theoretical evidence
reemphasizes the need for reliable descriptions of uncertain-
ties in both the forcing and response data if a meaningful
decomposition of predictive uncertainty is required [e.g.,
Huard and Mailhot, 2008; Renard et al., 2010]. Since the
inference can be sensitive to these specifications [Renard
et al., 2010; Weerts and El Serafy, 2006], using an unreli-
able error model will generally yield an unreliable uncer-
tainty decomposition. Hence, using literature values from
other studies may not always be adequate. For instance, rat-
ing curve errors depend on the hydraulic configuration of
the gauging section, the number of stage-discharge meas-
urements, the degree of extrapolation, etc., all of which are
site specific. Similarly, structural errors of a hydrological
model are likely to depend on the catchment, time period,
etc., and are difficult to estimate a priori.
[
18] An alternative to fixing the error model parameters
a priori is to include them in the inference. For instance,
the variance of rainfall errors can be estimated during
hydrological model calibration, rather than being fixed a
priori. Although this distinction may appear a superficial
technicality, it is highly pertinent to the inference in the
presence of multiple sources of errors [Huard and Mailhot,
2008; Renard et al., 2010; Weerts and El Serafy, 2006]. In
particular, fixing the error model parameters to incorrect
W11516 RENARD ET AL.: DECOMPOSING PREDICTIVE UNCERTAINTY IN HYDROLOGICAL MODELING W11516
2of21

values may yield a computationally tractable, yet statisti-
cally unreliable inference. On the other hand, the informa-
tion content of the data may not be sufficient to support the
inference of the error model parameters.
[
19] The approach of inferring the error model parame-
ters was used in the studies of Kavetski et al. [2006c],
Reichert and Mieleitner [2009], and Thyer et al. [2009].
However, these studies did not attempt to fully decompose
predictive uncertainty. Kuczera et al. [2006] attempted to
simultaneously infer rainfall and structural errors but lim-
ited themselves to point estimates of inferred quantities,
thus leaving open questions regarding parameter identifi-
ability and posterior well posedness. More recently, Renard
et al. [2010] and Kuczera et al. [2010b] quantitatively dem-
onstrated the difficulties of simultaneously identifying rain-
fall and structural errors from rainfall-runoff data when
only vague estimates of data uncertainty are known prior to
the hydrological model calibration. This result confirms the
earlier discussions by Beven [2005, 2006] of potential inter-
actions between multiple sources of error. However,
Renard et al. [2010] also illustrated that the use of more
precise (though still inexact) statistical descriptions of data
errors makes the posterior distribution well posed.
[
20] It is therefore vital that priors on individual sources
of error reflect actual knowledge, rather than be used as
mere numerical tricks to achieve well posedness. Given the
difficulty of obtaining prior estimates of structural errors
(especially for highly conceptualized rainfall-runoff mod-
els), it may be more practical to first focus on the observatio-
nal uncertainty in the rainfall-runoff data. Provided the data
error models are reliable, they can achieve closure on the
total errors and can allow reliably estimating structural
errors as ‘what remains once data errors are accounted for.
1.4. Study Aims
[
21] The aims of this paper are the following: (1) dem-
onstrate the development and incorporation of uncertainty
models for forcing and response data into a Bayesian meth-
odology for hydrological calibration and prediction, (2)
examine the resulting improvements in the predictive per-
formance, (3) evaluate whether using informative models
for data errors enables inference of structural errors as part
of the model calibration process, and (4) evaluate the abil-
ity of the inference to provide quantitative insights into the
relative contributions of individual sources of uncertainty.
Point 3 is of primary importance because of the intrinsic
difficulty in defining structural error models a priori. This
constitutes a major contribution of this paper since previous
attempts at isolating the contribution of structural errors to
predictive uncertainty [Huard and Mailhot, 2008; Salamon
and Feyen, 2010] were based on assuming known parame-
ters of the structural error model.
[
22] This paper uses the Bayesian total error analysis
(BATEA) [Kavetski et al., 2002, 2006b; Kuczera et al.,
2006]. The Bayesian foundation of BATEA, in particular,
its ability to exploit quantitative (though potentially vague)
probabilistic insights into individual sources of error, makes
it well suited for using independent knowledge to improve
parameter inference and predictions and to quantify indi-
vidual contributions to predictive uncertainties. However,
the development of realistic error models for rainfall and
runoff errors is of general interest for any method aiming at
decomposing the predictive uncertainty into its three main
contributive sources.
[
23] Here the rainfall error model is developed using a geo-
statistical analysis of the rain gauge network coupled with
condition simulation (CS) [e.g., Vischel et al., 2009]. For the
runoff data, the rating curve data and stage-discharge meas-
urements are used to derive a heteroscedastic error model
[Thyer et al., 2009]. The BATEA framework is then used to
explore different calibration schemes for integrating observa-
tional uncertainty into the inference and to evaluate their influ-
ence on calibration and validation, focusing on objectives 2–4.
[
24]Thisworkisinnovativeinseveralaspects.First,
while the characterization of rainfall errors has received
considerable attention [e.g., Krajewski et al.,2003;Villarini
et al., 2008], a comprehensive integration of this knowledge
within a Bayesian statistical inference for hydrological mod-
els has yet to be demonstrated in a real catchment case study.
More generally, the integration of independently derived
data error models into a Bayesian framework for probabilis-
tic predictions and a stringent verification and refinement of
all error models are of increasing interest not just in hydrol-
ogy but elsewhere in environmental sciences [e.g., Cressie
et al., 2009]. Finally, a systematic disaggregation of predic-
tive uncertainty into its contributing components in realistic
case studies is only in its nascence. Previous studies in this
area [e.g., Huard and Mailhot,2008;Salamon and Feyen,
2010] were based on assuming known fixed values for the
structural error parameters, which is hardly tenable, as dis-
cussed in section 1.3.
[
25] Second, this study further develops the BATEA
approach. Previous applications of BATEA focused primar-
ily on rainfall errors and lacked a separate characterization
of structural errors [Kavetski et al., 2006a; Thyer et al.,
2009]. Kuczera et al. [2006] explored separate specifica-
tions of rainfall, runoff, and structural errors but did not use
informative priors on the parameters of their error models
nor carried out a full Bayesian treatment of the posterior
distribution (they limited themselves to finding the poste-
rior mode only). Renard et al. [2010] illustrated, on the ba-
sis of synthetic experiments, the necessity of deriving
reliable and precise prior descriptions of data errors to
achieve well-posed inferences. The present paper builds on
the latter work and proposes a practical strategy toward
these objectives. Moreover, it explicitly demonstrates the
utility of independent rainfall error analysis for improving
the predictive reliability and for gaining quantitative and
qualitative insights into the contribution of different sour-
ces of errors in hydrological prediction.
1.5. Outline of Presentation
[
26] The Bayesian inference framework is outlined in
section 2. Section 3 describes the specific data and methods
used in this case study: the hydrological model and catch-
ment data are described in section 3.1; section 3.2
describes the geostatistical rain gauge analysis, the devel-
opment of an error model for the catchment average rainfall
data, and its incorporation into the Bayesian inference; sec-
tion 3.3 describes the runoff error model, and section 3.4
discusses the treatment of structural errors. Section 4
presents the results of a case study that evaluates the utility
of this information in improving the quantification and
decomposition of the runoff predictive uncertainty, with an
W11516 RENARD ET AL.: DECOMPOSING PREDICTIVE UNCERTAINTY IN HYDROLOGICAL MODELING W11516
3of21

emphasis on posterior scrutiny of the hypotheses made dur-
ing calibration. The results are discussed in section 5, fol-
lowed by a summary of key conclusions in section 6.
2. Theory: Bayesian Framework
2.1. General Setup: Data and Model
[
27] In general, a rainfall-runoff (RR) model, H()
hypothesizes a mapping between rainfall and runoff, given
a set of (usually time invariant) parameters h. Let R ¼
R
1:N
t
¼ðR
t
Þ
t¼1; ... ; N
t
and Q ¼ Q
1:N
t
¼ðQ
t
Þ
t¼1; ... ; N
t
denote
the true rainfall and true runoff time series of length N
t
,
respectively. Let
^
Q denote the runoff predicted by the RR
model, such that
^
Q ¼ HðR; hÞ: ð1Þ
[28] Hydrological models are usually also forced with
potential evapotranspiration (PET). However, sensitivity to
PET random errors is minor, and the impact of PET sys-
tematic errors remains much smaller than that of rainfall
errors [e.g., Oudin et al., 2006]. We therefore exclude PET
uncertainty from the analysis and notation. The influence
of initial conditions is minimized using a warm-up.
2.2. Data Uncertainty
[
29] The uncertainty in the rainfall-runoff data can be
characterized using statistical error models, which describe
what is known about the true values given the observations,
R pðRj
~
R; H
R
Þð2Þ
Q pðQj
^
Q; H
Q
Þ; ð3Þ
where H
R
and H
Q
are error model parameters describing
the statistical properties of the rainfall and runoff errors,
respectively (e.g., means, variances, and autocorrelations
of observation errors). The specification of these error mod-
els is a major focus of this paper. It will be described in
detail in sections 3.2 (rainfall) and 3.3 (runoff).
2.3. Structural Errors of Rainfall-Runoff Models
[
30] Unlike data errors, which can be estimated inde-
pendently from the hydrological model by analyzing the
observational network, no widely accepted approaches
exist for characterizing structural uncertainty [e.g., see
Beven, 2005, 2006; Doherty and Welter, 2010; Kennedy
and O’Hagan, 2001; Kuczera et al., 2006]. The most com-
mon approach is to use an exogenous structural error term
[e.g., Huard and Mailhot, 2008; Kavetski et al., 2006b]
Q ¼
^
Q þ n ¼ HðR; hÞþn ð4Þ
n pðnjH
Þ; ð5Þ
where n is an additive error. For instance, standard least
squares regression corresponds to assuming n Nðnj0;
2
Þ
and assuming that this term also accounts for input and out-
put errors.
[
31] A more recent strategy seeks to represent structural
uncertainty as a stochastic variation of one or more RR
model parameters [e.g., Kuczera et al., 2006; Rajaram and
Georgakakos, 1989; Reichert and Mieleitner, 2009; Smith
et al., 2008] or states [e.g., Moradkhani et al., 2005a,
2005b, 2006; Salamon and Feyen, 2009; Vrugt et al.,
2005; Weerts and El Serafy, 2006]. Time- and state-varying
parameters have also been explored within the instrumental
variable literature [e.g., Young, 1998; Young et al., 2001].
[
32] In this paper, we use a hierarchical structural error
model that hypothesizes a single stochastic RR parameter
K, which varies on a characteristic time scale represented
using epochs !,
^
Q
t
¼ HðR
1:t
; K
1:!ðtÞ
; hÞð6Þ
!ðtÞ
pðj
Þ; ð7Þ
where !ðtÞ is the epoch associated with the tth time step
and H
are parameters describing the statistical properties
of the stochastic parameters (e.g., H
could contain the
mean and variance of storm-dependent parameters).
[
33] A key challenge in using approach (7) is the mean-
ingful specification of H
. Since structural error remains
the least understood source of uncertainty, scarce guidance
exists for specifying anything other than vague priors,
whether on exogenous structural error terms or on stochas-
tic parameters.
2.4. Remnant Errors
[
34] In addition to error models developed for particular
error sources, we also account for ‘remnant’ errors
[Renard et al., 2010; Thyer et al., 2009]. These are related
to the notions of ‘model inadequacy’ [Kennedy and
O’Hagan, 2001] and ‘model discrepancy’ [Goldstein and
Rougier, 2009] but are intended to capture not only unac-
counted structural errors of the hydrological model but also
inevitable imperfections and omissions in the descriptions
of data uncertainty.
[
35] Here we assume additive Gaussian remnant errors "
t
with unknown variance
2
"
,
Q
t
¼
^
Q
t
þ "
t
; "
t
Nð0;
2
"
Þ: ð8Þ
[36] Note that in traditional regression, remnant errors
such as (8) represent the lumped effects of all sources of
error and correspond to ‘residual’ errors.
2.5. Posterior Distribution
[
37] When derived using the approach of Kavetski et al.
[2002] and Kuczera et al. [2010b], the BATEA posterior
distribution is given by Bayes’ theorem as follows:
pðh;R;K;Hj
~
R;
~
QÞ¼pð
~
R;
~
Qjh;R;K;HÞpðh;R;K;HÞ=pð
~
R;
~
QÞ; ð9Þ
pðh;R;K;Hj
~
R;
~
QÞ/pð
~
Qjh;K; R;H
Q
;H
"
Þpð
~
RjR;H
R
ÞpðKjH
Þ
pðRÞpðH
R
ÞpðH
Q
ÞpðH
ÞpðH
"
ÞpðhÞ:
ð10Þ
[38] The BATEA posterior in equation (10) explicitly
represents individual sources of uncertainty in the hydro-
logical model-data system as follows.
W11516 RENARD ET AL.: DECOMPOSING PREDICTIVE UNCERTAINTY IN HYDROLOGICAL MODELING W11516
4of21

Citations
More filters
Journal ArticleDOI
TL;DR: The Prediction in Ungauged Basins (PUB) initiative of the International Association of Hydrological Sciences (IAHS) launched in 2003 and concluded by the PUB Symposium 2012 held in Delft (23-25 October 2012), set out to shift the scientific culture of hydrology towards improved scientific understanding of hydrological processes, as well as associated uncertainties and the development of models with increasing realism and predictive power as discussed by the authors.
Abstract: The Prediction in Ungauged Basins (PUB) initiative of the International Association of Hydrological Sciences (IAHS), launched in 2003 and concluded by the PUB Symposium 2012 held in Delft (23–25 October 2012), set out to shift the scientific culture of hydrology towards improved scientific understanding of hydrological processes, as well as associated uncertainties and the development of models with increasing realism and predictive power. This paper reviews the work that has been done under the six science themes of the PUB Decade and outlines the challenges ahead for the hydrological sciences community.Editor D. KoutsoyiannisCitation Hrachowitz, M., Savenije, H.H.G., Bloschl, G., McDonnell, J.J., Sivapalan, M., Pomeroy, J.W., Arheimer, B., Blume, T., Clark, M.P., Ehret, U., Fenicia, F., Freer, J.E., Gelfan, A., Gupta, H.V., Hughes, D.A., Hut, R.W., Montanari, A., Pande, S., Tetzlaff, D., Troch, P.A., Uhlenbrook, S., Wagener, T., Winsemius, H.C., Woods, R.A., Zehe, E., and Cudennec, C., 2013. A d...

848 citations

Journal ArticleDOI
TL;DR: The basic theory of Markov chain Monte Carlo (MCMC) simulation is reviewed and a MATLAB toolbox of the DiffeRential Evolution Adaptive Metropolis (DREAM) algorithm developed by Vrugt et al. is introduced, used for Bayesian inference in fields ranging from physics, chemistry and engineering, to ecology, hydrology, and geophysics.
Abstract: Bayesian inference has found widespread application and use in science and engineering to reconcile Earth system models with data, including prediction in space (interpolation), prediction in time (forecasting), assimilation of observations and deterministic/stochastic model output, and inference of the model parameters. Bayes theorem states that the posterior probability, p ( H | Y ˜ ) of a hypothesis, H is proportional to the product of the prior probability, p(H) of this hypothesis and the likelihood, L ( H | Y ˜ ) of the same hypothesis given the new observations, Y ˜ , or p ( H | Y ˜ ) ∝ p ( H ) L ( H | Y ˜ ) . In science and engineering, H often constitutes some numerical model, ℱ(x) which summarizes, in algebraic and differential equations, state variables and fluxes, all knowledge of the system of interest, and the unknown parameter values, x are subject to inference using the data Y ˜ . Unfortunately, for complex system models the posterior distribution is often high dimensional and analytically intractable, and sampling methods are required to approximate the target. In this paper I review the basic theory of Markov chain Monte Carlo (MCMC) simulation and introduce a MATLAB toolbox of the DiffeRential Evolution Adaptive Metropolis (DREAM) algorithm developed by Vrugt et al. (2008a, 2009a) and used for Bayesian inference in fields ranging from physics, chemistry and engineering, to ecology, hydrology, and geophysics. This MATLAB toolbox provides scientists and engineers with an arsenal of options and utilities to solve posterior sampling problems involving (among others) bimodality, high-dimensionality, summary statistics, bounded parameter spaces, dynamic simulation models, formal/informal likelihood functions (GLUE), diagnostic model evaluation, data assimilation, Bayesian model averaging, distributed computation, and informative/noninformative prior distributions. The DREAM toolbox supports parallel computing and includes tools for convergence analysis of the sampled chain trajectories and post-processing of the results. Seven different case studies illustrate the main capabilities and functionalities of the MATLAB toolbox.

521 citations


Cites methods from "Toward a reliable decomposition of ..."

  • ...Latent variables can also be used to augment likelihood functions and take better consideration of forcing data and model structural error (Kavetski et al., 2006a; Vrugt et al., 2008a; Renard et al., 2011)....

    [...]

Journal ArticleDOI
TL;DR: Future EA-based applications to real-world problems require a fundamental shift of focus towards improving problem formulations, understanding general theoretic frameworks for problem decompositions, major advances in EA computational efficiency, and most importantly aiding real decision-making in complex, uncertain application contexts.
Abstract: The development and application of evolutionary algorithms (EAs) and other metaheuristics for the optimisation of water resources systems has been an active research field for over two decades. Research to date has emphasized algorithmic improvements and individual applications in specific areas (e.g. model calibration, water distribution systems, groundwater management, river-basin planning and management, etc.). However, there has been limited synthesis between shared problem traits, common EA challenges, and needed advances across major applications. This paper clarifies the current status and future research directions for better solving key water resources problems using EAs. Advances in understanding fitness landscape properties and their effects on algorithm performance are critical. Future EA-based applications to real-world problems require a fundamental shift of focus towards improving problem formulations, understanding general theoretic frameworks for problem decompositions, major advances in EA computational efficiency, and most importantly aiding real decision-making in complex, uncertain application contexts.

516 citations

Journal ArticleDOI
Abstract: This review and commentary sets out the need for authoritative and concise information on the expected error distributions and magnitudes in observational data We discuss the necessary components of a benchmark of dominant data uncertainties and the recent developments in hydrology which increase the need for such guidance We initiate the creation of a catalogue of accessible information on characteristics of data uncertainty for the key hydrological variables of rainfall, river discharge and water quality (suspended solids, phosphorus and nitrogen) This includes demonstration of how uncertainties can be quantified, summarizing current knowledge and the standard quantitative results available In particular, synthesis of results from multiple studies allows conclusions to be drawn on factors which control the magnitude of data uncertainty and hence improves provision of prior guidance on those uncertainties Rainfall uncertainties were found to be driven by spatial scale, whereas river discharge uncertainty was dominated by flow condition and gauging method Water quality variables presented a more complex picture with many component errors For all variables, it was easy to find examples where relative error magnitudes exceeded 40% We consider how data uncertainties impact on the interpretation of catchment dynamics, model regionalization and model evaluation In closing the review, we make recommendations for future research priorities in quantifying data uncertainty and highlight the need for an improved ‘culture of engagement’ with observational uncertainties Copyright © 2012 John Wiley & Sons, Ltd

315 citations

Journal ArticleDOI
TL;DR: The Dominant river tracing-Routing Integrated with Variable Infiltration Capacity (VIC) model is coupled with a newly developed hierarchical dominant river tracing based runoff-routing model to form a new core of the real-time Global Flood Monitoring System (GFMS) as discussed by the authors.
Abstract: A widely used land surface model, the Variable Infiltration Capacity (VIC) model, is coupled with a newly developed hierarchical dominant river tracing-based runoff-routing model to form the Dominant river tracing-Routing Integrated with VIC Environment (DRIVE) model, which serves as the new core of the real-time Global Flood Monitoring System (GFMS). The GFMS uses real-time satellite-based precipitation to derive flood monitoring parameters for the latitude band 50 deg. N - 50 deg. S at relatively high spatial (approximately 12 km) and temporal (3 hourly) resolution. Examples of model results for recent flood events are computed using the real-time GFMS (http://flood.umd.edu). To evaluate the accuracy of the new GFMS, the DRIVE model is run retrospectively for 15 years using both research-quality and real-time satellite precipitation products. Evaluation results are slightly better for the research-quality input and significantly better for longer duration events (3 day events versus 1 day events). Basins with fewer dams tend to provide lower false alarm ratios. For events longer than three days in areas with few dams, the probability of detection is approximately 0.9 and the false alarm ratio is approximately 0.6. In general, these statistical results are better than those of the previous system. Streamflow was evaluated at 1121 river gauges across the quasi-global domain. Validation using real-time precipitation across the tropics (30 deg. S - 30 deg. N) gives positive daily Nash-Sutcliffe Coefficients for 107 out of 375 (28%) stations with a mean of 0.19 and 51% of the same gauges at monthly scale with a mean of 0.33. There were poorer results in higher latitudes, probably due to larger errors in the satellite precipitation input.

280 citations


Cites methods from "Toward a reliable decomposition of ..."

  • ...Further work is needed to develop techniques or deploy existing ones from the literature [e.g., Beven and Freer, 2001; Renard et al., 2011; Demirel et al., 2013] for systematic uncertainty analysis....

    [...]

References
More filters
Journal ArticleDOI
TL;DR: In this paper, the authors consider the problem of comparing complex hierarchical models in which the number of parameters is not clearly defined and derive a measure pD for the effective number in a model as the difference between the posterior mean of the deviances and the deviance at the posterior means of the parameters of interest, which is related to other information criteria and has an approximate decision theoretic justification.
Abstract: Summary. We consider the problem of comparing complex hierarchical models in which the number of parameters is not clearly defined. Using an information theoretic argument we derive a measure pD for the effective number of parameters in a model as the difference between the posterior mean of the deviance and the deviance at the posterior means of the parameters of interest. In general pD approximately corresponds to the trace of the product of Fisher's information and the posterior covariance, which in normal models is the trace of the ‘hat’ matrix projecting observations onto fitted values. Its properties in exponential families are explored. The posterior mean deviance is suggested as a Bayesian measure of fit or adequacy, and the contributions of individual observations to the fit and complexity can give rise to a diagnostic plot of deviance residuals against leverages. Adding pD to the posterior mean deviance gives a deviance information criterion for comparing models, which is related to other information criteria and has an approximate decision theoretic justification. The procedure is illustrated in some examples, and comparisons are drawn with alternative Bayesian and classical proposals. Throughout it is emphasized that the quantities required are trivial to compute in a Markov chain Monte Carlo analysis.

11,691 citations

Journal ArticleDOI
TL;DR: In this article, a new sequential data assimilation method is proposed based on Monte Carlo methods, a better alternative than solving the traditional and computationally extremely demanding approximate error covariance equation used in the extended Kalman filter.
Abstract: A new sequential data assimilation method is discussed. It is based on forecasting the error statistics using Monte Carlo methods, a better alternative than solving the traditional and computationally extremely demanding approximate error covariance equation used in the extended Kalman filter. The unbounded error growth found in the extended Kalman filter, which is caused by an overly simplified closure in the error covariance equation, is completely eliminated. Open boundaries can be handled as long as the ocean model is well posed. Well-known numerical instabilities associated with the error covariance equation are avoided because storage and evolution of the error covariance matrix itself are not needed. The results are also better than what is provided by the extended Kalman filter since there is no closure problem and the quality of the forecast error statistics therefore improves. The method should be feasible also for more sophisticated primitive equation models. The computational load for reasonable accuracy is only a fraction of what is required for the extended Kalman filter and is given by the storage of, say, 100 model states for an ensemble size of 100 and thus CPU requirements of the order of the cost of 100 model integrations. The proposed method can therefore be used with realistic nonlinear ocean models on large domains on existing computers, and it is also well suited for parallel computers and clusters of workstations where each processor integrates a few members of the ensemble.

4,816 citations


"Toward a reliable decomposition of ..." refers methods in this paper

  • ...Multiple studies have employed recursive data assimilation methods such as extended and ensemble Kalman filters [Evensen, 1994; Moradkhani et al., 2005b; Rajaram and Georgakakos, 1989; Reichle et al., 2002; Vrugt et al., 2005] or Bayesian filtering [Moradkhani et al., 2005a, 2006; Salamon and…...

    [...]

  • ...Multiple studies have employed recursive data assimilation methods such as extended and ensemble Kalman filters [Evensen, 1994; Moradkhani et al., 2005b; Rajaram and Georgakakos, 1989; Reichle et al., 2002; Vrugt et al., 2005] or Bayesian filtering [Moradkhani et al....

    [...]

Journal ArticleDOI
TL;DR: A Bayesian calibration technique which improves on this traditional approach in two respects and attempts to correct for any inadequacy of the model which is revealed by a discrepancy between the observed data and the model predictions from even the best‐fitting parameter values is presented.
Abstract: We consider prediction and uncertainty analysis for systems which are approximated using complex mathematical models. Such models, implemented as computer codes, are often generic in the sense that by a suitable choice of some of the model's input parameters the code can be used to predict the behaviour of the system in a variety of specific applications. However, in any specific application the values of necessary parameters may be unknown. In this case, physical observations of the system in the specific context are used to learn about the unknown parameters. The process of fitting the model to the observed data by adjusting the parameters is known as calibration. Calibration is typically effected by ad hoc fitting, and after calibration the model is used, with the fitted input values, to predict the future behaviour of the system. We present a Bayesian calibration technique which improves on this traditional approach in two respects. First, the predictions allow for all sources of uncertainty, including the remaining uncertainty over the fitted parameters. Second, they attempt to correct for any inadequacy of the model which is revealed by a discrepancy between the observed data and the model predictions from even the best-fitting parameter values. The method is illustrated by using data from a nuclear radiation release at Tomsk, and from a more complex simulated nuclear accident exercise.

3,745 citations


"Toward a reliable decomposition of ..." refers background or methods in this paper

  • ...[30] Unlike data errors, which can be estimated independently from the hydrological model by analyzing the observational network, no widely accepted approaches exist for characterizing structural uncertainty [e.g., see Beven, 2005, 2006; Doherty and Welter, 2010; Kennedy and O’Hagan, 2001; Kuczera et al., 2006]....

    [...]

  • ...These are related to the notions of ‘‘model inadequacy’’ [Kennedy and O’Hagan, 2001] and ‘‘model discrepancy’’ [Goldstein and Rougier, 2009] but are intended to capture not only unaccounted structural errors of the hydrological model but also inevitable imperfections and omissions in the…...

    [...]

  • ...These are related to the notions of ‘‘model inadequacy’’ [Kennedy and O’Hagan, 2001] and ‘‘model discrepancy’’ [Goldstein and Rougier, 2009] but are intended to capture not only unaccounted structural errors of the hydrological model but also inevitable imperfections and omissions in the descriptions of data uncertainty....

    [...]

  • ...…errors, which can be estimated independently from the hydrological model by analyzing the observational network, no widely accepted approaches exist for characterizing structural uncertainty [e.g., see Beven, 2005, 2006; Doherty and Welter, 2010; Kennedy and O’Hagan, 2001; Kuczera et al., 2006]....

    [...]

Journal ArticleDOI
Keith Beven1
TL;DR: The argument is made that the potential for multiple acceptable models as representations of hydrological and other environmental systems (the equifinality thesis) should be given more serious consideration than hitherto.

2,073 citations


"Toward a reliable decomposition of ..." refers background in this paper

  • ...…errors, which can be estimated independently from the hydrological model by analyzing the observational network, no widely accepted approaches exist for characterizing structural uncertainty [e.g., see Beven, 2005, 2006; Doherty and Welter, 2010; Kennedy and O’Hagan, 2001; Kuczera et al., 2006]....

    [...]

Journal ArticleDOI
TL;DR: All of the methods in this work can fail to detect the sorts of convergence failure that they were designed to identify, so a combination of strategies aimed at evaluating and accelerating MCMC sampler convergence are recommended.
Abstract: A critical issue for users of Markov chain Monte Carlo (MCMC) methods in applications is how to determine when it is safe to stop sampling and use the samples to estimate characteristics of the distribution of interest. Research into methods of computing theoretical convergence bounds holds promise for the future but to date has yielded relatively little of practical use in applied work. Consequently, most MCMC users address the convergence problem by applying diagnostic tools to the output produced by running their samplers. After giving a brief overview of the area, we provide an expository review of 13 convergence diagnostics, describing the theoretical basis and practical implementation of each. We then compare their performance in two simple models and conclude that all of the methods can fail to detect the sorts of convergence failure that they were designed to identify. We thus recommend a combination of strategies aimed at evaluating and accelerating MCMC sampler convergence, including ap...

1,860 citations

Frequently Asked Questions (11)
Q1. What have the authors contributed in "Toward a reliable decomposition of predictive uncertainty in hydrological modeling: characterizing rainfall errors using conditional simulation" ?

The electronic version of this article is the complete one and can be found online at: http: //onlinelibrary. 

Other areas in need of further research attention include the following. [ 140 ] In particular, the use of structural errors to diagnose, compare, or improve hydrological models remains an important area of future research [ e. 2. The stochastic parameter approach needs further appraisal and more informative structural error models should be developed. 4. The understanding of structural errors can be improved. 

Because of the short length of the densely gauged R13H rainfall time series for this catchment, it was used entirely to construct the rainfall error model for the calibration period and was not used to check the rainfall PD in the validation period. 

The rating curve error model needs to be generalized to rigorously distinguish between random and systematic rating curve errors and to account for their likely autocorrelation at short time scales. 

Direct validation of the estimated structural uncertainty requires highly accurate forcing and response data, so that structural errors can be isolated. 

Future avenues for scrutinizing the rainfall component include comparing inferred rainfall errors with the errors suggested by other sources, such as radar. 

Given the manifest significance of a robust quantitative understanding of data and modeling uncertainties in environmental studies, further development and implementation of instrumental and statistical procedures is needed to estimate the accuracy and precision of environmental data at the data collection and postprocessing stages. 

The discrepancies in small rainfall events have several possible explanations: (1) biases in the R3D areal averages due to insufficient spatial coverage and/or (2) biases in the CS of small rainfall events. 

Since these correspond to conditional distributions (see section 2.7), the choice of the conditioning values may affect the decomposition of uncertainty. 

It is therefore vital that priors on individual sources of error reflect actual knowledge, rather than be used as mere numerical tricks to achieve well posedness. 

It is therefore vital that priors on individual sources of error reflect actual knowledge, rather than be used as mere numerical tricks to achieve well posedness.