Toward a reliable decomposition of predictive uncertainty in hydrological modeling: Characterizing rainfall errors using conditional simulation
Summary (5 min read)
1.1. Hydrological Modeling in the Presence of Rainfall and Runoff Errors
- [2] Data and model errors conspire to make reliable and robust calibration of hydrological models a difficult task.
- The rain gauges themselves are subject to both systematic and random measurement errors, including mechanical limitations, wind effects, and evaporation losses, all of which are design specific and can vary substantially with rainfall intensity [Molini et al., 2005].
- Finally, the characterization of structural uncertainty is a particularly challenging task, and the hydrological community has yet to agree on suitable definitions and approaches for handling structural model errors in the 1UR HHLY Hydrology-Hydraulics, Cemagref, Lyon, France.
1.2. Decomposing Predictive Uncertainty
- The focus of this paper is on the decomposition of the total uncertainty in hydrological predictions into its contributing sources.
- It helps in more informed research and experimental resource allocation, and, importantly, allow a meaningful a posteriori evaluation of these efforts. [11].
- In a prediction context, attempts to decompose the total uncertainty into its three main sources have been made using several related methods.
- The authors consider Bayesian hierarchical approaches [e.g., Huard and Mailhot, 2008; Kuczera et al., 2006], which to date have been implemented in batch estimation form (but can also be formulated recursively).
1.3. Specifying Data and Structural Error Models
- Output, and structural errors is well known, developing quantitative error models is a considerable challenge in hydrological applications.
- Similarly, Salamon and Feyen [2010] used literature values for runoff errors ( 12.5% standard error for large runoff) and rule-of-thumb values for rainfall and structural errors ( 15% standard error). [17].
- These studies did not attempt to fully decompose predictive uncertainty.
- More recently, Renard et al. [2010] and Kuczera et al. [2010b] quantitatively demonstrated the difficulties of simultaneously identifying rainfall and structural errors from rainfall-runoff data when only vague estimates of data uncertainty are known prior to the hydrological model calibration.
- This result confirms the earlier discussions by Beven [2005, 2006] of potential interactions between multiple sources of error.
1.4. Study Aims
- This constitutes a major contribution of this paper since previous attempts at isolating the contribution of structural errors to predictive uncertainty [Huard and Mailhot, 2008; Salamon and Feyen, 2010] were based on assuming known parameters of the structural error model. [22].
- The Bayesian foundation of BATEA, in particular, its ability to exploit quantitative (though potentially vague) probabilistic insights into individual sources of error, makes it well suited for using independent knowledge to improve parameter inference and predictions and to quantify individual contributions to predictive uncertainties.
- The development of realistic error models for rainfall and runoff errors is of general interest for any method aiming at decomposing the predictive uncertainty into its three main contributive sources. [23].
- This work is innovative in several aspects.
2.1. General Setup: Data and Model
- Nt denote the true rainfall and true runoff time series of length Nt, respectively.
- Hydrological models are usually also forced with potential evapotranspiration (PET).
- Sensitivity to PET random errors is minor, and the impact of PET systematic errors remains much smaller than that of rainfall errors [e.g., Oudin et al., 2006].
- The authors therefore exclude PET uncertainty from the analysis and notation.
2.3. Structural Errors of Rainfall-Runoff Models
- Unlike data errors, which can be estimated independently from the hydrological model by analyzing the observational network, no widely accepted approaches exist for characterizing structural uncertainty [e.g., see Beven, 2005, 2006; Doherty and Welter, 2010; Kennedy and O’Hagan, 2001; Kuczera et al., 2006].
- Standard least squares regression corresponds to assuming n Nðnj0; 2 Þ and assuming that this term also accounts for input and output errors. [31].
- Since structural error remains the least understood source of uncertainty, scarce guidance exists for specifying anything other than vague priors, whether on exogenous structural error terms or on stochastic parameters.
2.4. Remnant Errors
- In addition to error models developed for particular error sources, the authors also account for ‘‘remnant’’ errors [Renard et al., 2010; Thyer et al., 2009].
- These are related to the notions of ‘‘model inadequacy’’ [Kennedy and O’Hagan, 2001] and ‘‘model discrepancy’’ [Goldstein and Rougier, 2009] but are intended to capture not only unaccounted structural errors of the hydrological model but also inevitable imperfections and omissions in the descriptions of data uncertainty. [35].
- Note that in traditional regression, remnant errors such as (8) represent the lumped effects of all sources of error and correspond to ‘‘residual’’ errors.
2.5. Posterior Distribution
- The ‘‘rainfall likelihood’’ pð~RjR;HRÞ describes rainfalls errors. [41].
- The posterior in equation (10) can be explored using Markov chain Monte Carlo (MCMC) sampling.
- This would be needed, for example, if BATEA were applied recursively as new data arrives. [44].
- The key scientific (as opposed to computational) challenge in using BATEA or any other Bayesian approach for the decomposition of individual sources of error is to develop accurate and precise probabilistic models for the individual terms in the posterior (10).
2.6. Calibration Schemes
- The BATEA framework can be used to derive several calibration schemes, differing in the type of error models and the amount of prior knowledge utilized in the inference.
- The OI-CS scheme is an ‘‘enhanced’’ OI scheme, augmented using an informative prior for the term p(R).
- The output-input-structural (OIS) scheme explicitly accounts for rainfall and runoff uncertainty and characterizes structural errors using a stochastic RR parameter.
- Pr io r pð h Þ 5 of 21 it still uses the remnant error (8) to account for the inevitable imperfections of the uncertainty models. [50].
2.7.1. Total Predictive Distributions
- Let N denote the vector of all inferred quantities and pðNj~DÞ denote the posterior of parameters N given observed data ~D.
- The PPD pðY j~D; 1Þ, representing the uncertainty contributed by 2, can be defined in a similar manner. [56].
3.1.1. The Yzeron Catchment
- The case study is based on the 129 km2 Yzeron catchment in the Rhône-Alpes region of France, near Lyon .
- The annual average rainfall and runoff are approximately 845 and 150 mm respectively, yielding an annual runoff coefficient of 0.18.
- The first set, denoted as R3D, comprises three rain gauges in the lower areas of the catchment , with daily totals available for the whole period of study.
- This study applies the widely used GR4J model [Perrin et al., 2003], which simulates catchment runoff using rainfall and potential evapotranspiration at a daily time step .
3.2.1. Conditional Simulation
- The uncertainty of areal rainfall estimates is generally dominated by sampling errors, i.e., errors due to the incomplete description of the rainfall spatial field using rain gauges [Moulin et al., 2009; Severino and Alpuim, 2005].
- This allows the generated field to be consistent with the spatiotemporal properties of aggregated rainfall. [69].
- The discrepancies in small rainfall events have several possible explanations: (1) biases in the R3D areal averages due to insufficient spatial coverage and/or (2) biases in the CS of small rainfall events.
3.3. Development of the Runoff Error Model
- Runoff uncertainty was investigated by analyzing the rating curve and related stage-discharge measurements.
- Figure 6 shows the runoff measurement errors, defined as the difference between the runoff measurements and the runoff predicted by the rating curve (‘‘RCP runoff’’).
- Equation (17) was fitted to the Yzeron runoff data (with vague priors on a and b) using the WinBugs software [Spiegelhalter et al., 2003].
- This deficiency arises because of limited gauging data in the high-flow range (a single measurement for flows exceeding 10 mm). [89].
3.4. Representation of Structural Errors
- The characterization of structural error of the GR4J model is explored using stochastic daily variation of its parameter 1.
- The authors also investigate a more traditional exogenous treatment of structural errors using the remnant error term (see also section 2.4). [92].
- Note that 1 controls the maximum storage of the production store .
- A separate sensitivity analysis indicated that this parameter, when made stochastic, had the largest impact on model predictions.
- Importantly, the authors examined the inferred stochastic variability of 1 to determine its effect on the storage values and long-term water balance (section 4.4.2).
4.1. Well Posedness of the Calibration Schemes
- The convergence of MCMC samples reflects the statistical characteristics of the target distribution.
- In particular, slowly convergent sampling is often indicative of illposed posteriors [Renard et al., 2010].
- This emphasizes that the computational cost of an inference depends more on its structure than just on its dimensionality.
- Moreover, the posterior standard deviations of inferred quantities were higher than in the OIS-CS scheme by a factor of about 3 on average, but exceeding 10 for some latent variables.
- The nonconvergence of the OIS scheme, contrasted with the convergence of the OIS-CS scheme, supports a key conclusion of Renard et al. [2010], namely, that the specification of informative priors for rainfall and runoff uncertainty is a necessary step to ensure well posedness when both forcing and structural errors are modeled hierarchically using latent variables.
4.2. Reliability of Total Predictive Uncertainty (All Schemes)
- [99] Section 4.2 examines the adequacy of the predictive distribution of runoff.
- Moreover, the reliability of the total predictive distribution does not prove that all individual error models are correctly specified; it is a necessary but insufficient condition.
- The OI-CS scheme slightly underestimates the predictive uncertainty, with about 1% of the observations lying outside of the predictive range (p-values of 0 and 1 by convention).
- The OIS-CS scheme yields a reliable estimation of the predictive uncertainty for all runoff ranges .
4.3. Decomposition of Total Predictive Uncertainty Into Forcing, Response, and Structural Components (OIS-CS Only)
- [108] Section 4.2 showed that the BATEA methodology yields reliable estimates of predictive uncertainty when prior information on rainfall and runoff errors is available (OIS-CS scheme).
- Figure 9 shows the TPD and PPD for the forcing, structural, and response errors (see section 2.7 for details).
- Under the hypotheses made in this case study (including the hydrological model and the data error models), predictive uncertainty in the runoff appears to be dominated by structural errors.
- AThe posterior means are reported, followed by the corresponding posterior standard deviations.
- Note that this study uses partial predictive intervals in the decomposition of uncertainty.
4.4.1. Input Errors (OIS-CS Only)
- Figure 10 shows diagnostic plots to scrutinize the rainfall error model in equation (16).
- URFs for smaller rainfall events are highly variable, with some multipliers having a significant reduction in uncertainty after calibration.
- When the store exceeds its capacity, some water is subtracted from the production store.
- Accounting for data errors (OI-CS, OI, and OISCS schemes) markedly reduces the skewness and excess kurtosis of the standardized residuals.
- This further discredits the assumption of homoscedastic Gaussian remnant errors and needs to be addressed. [125].
5.1. Quantification of Predictive Uncertainty
- Section 4.2 indicated that the predictive distribution of runoff was fairly reliable for the OI-CS and OIS-CS scheme.
- It can be seen that the OI-CS scheme slightly underestimates predictive uncertainty , while the OI scheme yields significantly larger estimated input errors and predictive uncertainty.
- This suggests that the CS prior constrains the input error estimates and reduces their ability to interact with structural errors and compensate for unaccounted errors. [127].
- Importantly, the OIS scheme, which omits prior information on the rainfall errors, leads to an ill-posed inference (section 4.1).
- This is consistent with previous findings that priors on rainfall and runoff uncertainty control the well posedness of Bayesian hierarchical inferences in hydrology [Renard et al., 2010].
5.2. Decomposition of Predictive Uncertainty
- The empirical results in section 4.3 suggest that decomposing predictive uncertainty into its contributing sources is possible when independent estimates of rainfall and runoff data uncertainty are available and used in the BATEA inference.
- The reliability of this decomposition can be examined by considering (1) the reliability of the total predictive distribution in combination with (2) the reliability of the individual data and structural components.
- In applications where longer periods of densely gauged rainfall are available, it could be partitioned between calibration and validation. [130].
- Direct validation of the estimated structural uncertainty requires highly accurate forcing and response data, so that structural errors can be isolated.
5.3. On the Treatment of Structural Error
- The treatment of structural error remains a topic of active research (e.g., see the discussion by Beven [2005] and Doherty and Welter [2010]).
- This study does not aim to compare or improve methods for representing structural errors.
- Instead, it uses two particular structural error methods as part of a study pursuing error decomposition by exploiting independently derived data error models.
- The authors view this as a logical first step before structural error characterization is tackled. [133].
- Which of these approaches, if any, provide an adequate description of structural errors remains an open question.
6. Conclusions
- The application of the Bayesian framework in a real-data case study confirms earlier findings that prior information on data uncertainty is not merely beneficial but essential for a meaningful and reliable quantification and decomposition of the predictive uncertainty. [147].
- Conversely, including additional error models improved the reliability of the total uncertainty estimates.
- The authors stress that this improvement was demonstrated in the validation period and thus is unlikely to be due to potential overfitting. [149].
- Given the manifest significance of a robust quantitative understanding of data and modeling uncertainties in environmental studies, further development and implementation of instrumental and statistical procedures is needed to estimate the accuracy and precision of environmental data at the data collection and postprocessing stages.
Did you find this useful? Give us your feedback
Citations
848 citations
521 citations
Cites methods from "Toward a reliable decomposition of ..."
...Latent variables can also be used to augment likelihood functions and take better consideration of forcing data and model structural error (Kavetski et al., 2006a; Vrugt et al., 2008a; Renard et al., 2011)....
[...]
516 citations
315 citations
280 citations
Cites methods from "Toward a reliable decomposition of ..."
...Further work is needed to develop techniques or deploy existing ones from the literature [e.g., Beven and Freer, 2001; Renard et al., 2011; Demirel et al., 2013] for systematic uncertainty analysis....
[...]
References
11,691 citations
4,816 citations
"Toward a reliable decomposition of ..." refers methods in this paper
...Multiple studies have employed recursive data assimilation methods such as extended and ensemble Kalman filters [Evensen, 1994; Moradkhani et al., 2005b; Rajaram and Georgakakos, 1989; Reichle et al., 2002; Vrugt et al., 2005] or Bayesian filtering [Moradkhani et al., 2005a, 2006; Salamon and…...
[...]
...Multiple studies have employed recursive data assimilation methods such as extended and ensemble Kalman filters [Evensen, 1994; Moradkhani et al., 2005b; Rajaram and Georgakakos, 1989; Reichle et al., 2002; Vrugt et al., 2005] or Bayesian filtering [Moradkhani et al....
[...]
3,745 citations
"Toward a reliable decomposition of ..." refers background or methods in this paper
...[30] Unlike data errors, which can be estimated independently from the hydrological model by analyzing the observational network, no widely accepted approaches exist for characterizing structural uncertainty [e.g., see Beven, 2005, 2006; Doherty and Welter, 2010; Kennedy and O’Hagan, 2001; Kuczera et al., 2006]....
[...]
...These are related to the notions of ‘‘model inadequacy’’ [Kennedy and O’Hagan, 2001] and ‘‘model discrepancy’’ [Goldstein and Rougier, 2009] but are intended to capture not only unaccounted structural errors of the hydrological model but also inevitable imperfections and omissions in the…...
[...]
...These are related to the notions of ‘‘model inadequacy’’ [Kennedy and O’Hagan, 2001] and ‘‘model discrepancy’’ [Goldstein and Rougier, 2009] but are intended to capture not only unaccounted structural errors of the hydrological model but also inevitable imperfections and omissions in the descriptions of data uncertainty....
[...]
...…errors, which can be estimated independently from the hydrological model by analyzing the observational network, no widely accepted approaches exist for characterizing structural uncertainty [e.g., see Beven, 2005, 2006; Doherty and Welter, 2010; Kennedy and O’Hagan, 2001; Kuczera et al., 2006]....
[...]
2,073 citations
"Toward a reliable decomposition of ..." refers background in this paper
...…errors, which can be estimated independently from the hydrological model by analyzing the observational network, no widely accepted approaches exist for characterizing structural uncertainty [e.g., see Beven, 2005, 2006; Doherty and Welter, 2010; Kennedy and O’Hagan, 2001; Kuczera et al., 2006]....
[...]
1,860 citations
Related Papers (5)
Frequently Asked Questions (11)
Q2. What are the future works in "Toward a reliable decomposition of predictive uncertainty in hydrological modeling: characterizing rainfall errors using conditional simulation" ?
Other areas in need of further research attention include the following. [ 140 ] In particular, the use of structural errors to diagnose, compare, or improve hydrological models remains an important area of future research [ e. 2. The stochastic parameter approach needs further appraisal and more informative structural error models should be developed. 4. The understanding of structural errors can be improved.
Q3. Why was the rainfall error model used only partially?
Because of the short length of the densely gauged R13H rainfall time series for this catchment, it was used entirely to construct the rainfall error model for the calibration period and was not used to check the rainfall PD in the validation period.
Q4. What is the need for a more generalized rating curve error model?
The rating curve error model needs to be generalized to rigorously distinguish between random and systematic rating curve errors and to account for their likely autocorrelation at short time scales.
Q5. What is the way to evaluate the structural uncertainty?
Direct validation of the estimated structural uncertainty requires highly accurate forcing and response data, so that structural errors can be isolated.
Q6. What are the future avenues for scrutinizing the rainfall component?
Future avenues for scrutinizing the rainfall component include comparing inferred rainfall errors with the errors suggested by other sources, such as radar.
Q7. What is the significance of a robust quantitative understanding of data and modeling uncertainties in environmental studies?
Given the manifest significance of a robust quantitative understanding of data and modeling uncertainties in environmental studies, further development and implementation of instrumental and statistical procedures is needed to estimate the accuracy and precision of environmental data at the data collection and postprocessing stages.
Q8. What are the possible explanations for the discrepancies in small rainfall events?
The discrepancies in small rainfall events have several possible explanations: (1) biases in the R3D areal averages due to insufficient spatial coverage and/or (2) biases in the CS of small rainfall events.
Q9. What is the effect of the conditional distributions on the decomposition of uncertainty?
Since these correspond to conditional distributions (see section 2.7), the choice of the conditioning values may affect the decomposition of uncertainty.
Q10. Why is it important that priors on individual sources of error reflect actual knowledge?
It is therefore vital that priors on individual sources of error reflect actual knowledge, rather than be used as mere numerical tricks to achieve well posedness.
Q11. Why is it important that priors on individual sources of error reflect actual knowledge?
It is therefore vital that priors on individual sources of error reflect actual knowledge, rather than be used as mere numerical tricks to achieve well posedness.