Watershed models are powerful tools for simulating the effect of watershed processes and management on soil and water resources. However, no comprehensive guidance is available to facilitate model evaluation in terms of the accuracy of simulated data compared to measured flow and constituent values. Thus, the objectives of this research were to: (1) determine recommended model evaluation techniques (statistical and graphical), (2) review reported ranges of values and corresponding performance ratings for the recommended statistics, and (3) establish guidelines for model evaluation based on the review results and project-specific considerations; all of these objectives focus on simulation of streamflow and transport of sediment and nutrients. These objectives were achieved with a thorough review of relevant literature on model application and recommended model evaluation methods. Based on this analysis, we recommend that three quantitative statistics, Nash-Sutcliffe efficiency (NSE), percent bias (PBIAS), and ratio of the root mean square error to the standard deviation of measured data (RSR), in addition to the graphical techniques, be used in model evaluation. The following model evaluation performance ratings were established for each recommended statistic. In general, model simulation can be judged as satisfactory if NSE > 0.50 and RSR < 0.70, and if PBIAS + 25% for streamflow, PBIAS + 55% for sediment, and PBIAS + 70% for N and P. For PBIAS, constituent-specific performance ratings were determined based on uncertainty of measured data. Additional considerations related to model evaluation guidelines are also discussed. These considerations include: single-event simulation, quality and quantity of measured data, model calibration procedure, evaluation time step, and project scope and magnitude. A case study illustrating the application of the model evaluation guidelines is also provided.

Model Evaluation Guidelines for Systematic Quantification of Accuracy in Watershed Simulations

Statistics for Spatial Data.

The Tropical Rainfall Measuring Mission (TRMM) Multisatellite Precipitation Analysis (TMPA) provides a calibration-based sequential scheme for combining precipitation estimates from multiple satellites, as well as gauge analyses where feasible, at fine scales (0.25° × 0.25° and 3 hourly). TMPA is available both after and in real time, based on calibration by the TRMM Combined Instrument and TRMM Microwave Imager precipitation products, respectively. Only the after-real-time product incorporates gauge data at the present. The dataset covers the latitude band 50°N–S for the period from 1998 to the delayed present. Early validation results are as follows: the TMPA provides reasonable performance at monthly scales, although it is shown to have precipitation rate–dependent low bias due to lack of sensitivity to low precipitation rates over ocean in one of the input products [based on Advanced Microwave Sounding Unit-B (AMSU-B)]. At finer scales the TMPA is successful at approximately reproducing the s...

/pdf/the-trmm-multisatellite-precipitation-analysis-tmpa-quasi-3wulgfnahi.pdf

The TRMM Multisatellite Precipitation Analysis (TMPA): Quasi-Global, Multiyear, Combined-Sensor Precipitation Estimates at Fine Scales

List of Figures. List of Tables. Preface. Foreword. 1. Basic Concepts. 2. Evolutionary Algorithm MOP Approaches. 3. MOEA Test Suites. 4. MOEA Testing and Analysis. 5. MOEA Theory and Issues. 3. MOEA Theoretical Issues. 6. Applications. 7. MOEA Parallelization. 8. Multi-Criteria Decision Making. 9. Special Topics. 10. Epilog. Appendix A: MOEA Classification and Technique Analysis. Appendix B: MOPs in the Literature. Appendix C: Ptrue & PFtrue for Selected Numeric MOPs. Appendix D: Ptrue & PFtrue for Side-Constrained MOPs. Appendix E: MOEA Software Availability. Appendix F: MOEA-Related Information. Index. References.

/pdf/evolutionary-algorithms-for-solving-multi-objective-problems-2txxk4mqci.pdf

Evolutionary algorithms for solving multi-objective problems

Decomposition of the mean squared error and NSE performance criteria: Implications for improving hydrological modelling

Bayesian Recursive Estimation of Parameter and Output Uncertainty for Watershed Models

https://agupubs.onlinelibrary.wiley.com/doi/epdf/10.1029/2018WR022668

Revisiting the Basis of Sensitivity Analysis for Dynamical Earth System Models

[1] Modeling of natural systems typically involves conceptualization and parameterization to simplify the representations of the underlying process. Objective methods for estimation of the model parameters then require optimization of a cost function, representing a measure of distance between the observations and the corresponding model predictions, typically by calibration in a static batch mode and/or via some dynamic recursive optimization approach. Recently, there has been a focus on the development of parameter estimation methods that appropriately account for different sources of uncertainty. In this context, we introduce an approach to sample the optimal parameter space that uses nonparametric block bootstrapping coupled with global optimization. We demonstrate the applicability of this procedure via a case study, in which we estimate the parameter uncertainty resulting from uncertainty in the forcing data and evaluate its impacts on the resulting streamflow simulations.

/pdf/improving-robustness-of-hydrologic-parameter-estimation-by-1e4so34thc.pdf

Improving robustness of hydrologic parameter estimation by the use of moving block bootstrap resampling

There has been a recent resurgence of interest in the application of Information Theory to problems of system identification in the Earth and Environmental Sciences. While the concept of entropy has found increased application, little attention has yet been given to the practical problems of estimating entropy when dealing with the unique characteristics of two commonly used kinds of hydrologic data: rainfall and runoff. In this paper, we discuss four important issues of practical relevance that can bias the computation of entropy if not properly handled. The first (zero effect) arises when precipitation and ephemeral streamflow data must be viewed as arising from a discrete-continuous hybrid distribution due to the occurrence of many zero values (e.g., days with no rain/no runoff). Second, in the widely used bin-counting method for estimation of PDF's, significant error can be introduced if the bin width is not carefully selected. The third (measurement effect) arises due to the fact that continuously varying hydrologic variables can typically only be observed discretely to some degree of precision. The Fourth (skewness effect) arises when the distribution of a variable is significantly skewed. Here we present an approach that can deal with all four of these issues, and test them with artificially generated and real hydrological data. The results indicate that the method is accurate and robust.

https://agupubs.onlinelibrary.wiley.com/doi/pdf/10.1002/2014WR015874

Estimating information entropy for hydrological data: One‐dimensional case

[1] Understanding the relationship between computer-based models and the environment they simulate is becoming increasingly important as we try to predict how the earth's climate will change. As a surrogate for the representation of uncertainty in a prediction problem, it is common to use the range of behaviour from a set of models (an ensemble), and the ensemble mean as the ‘best guess’ prediction. We suggest a ‘model space’ metric, which, by providing one relevant definition of model independence, could allow us to begin to understand the relationship between model spread and prediction uncertainty. This in turn could allow the minimisation of bias from the inclusion of similar models in ensembles and quantification of how much independent information each model contributes to the prediction problem.

https://agupubs.onlinelibrary.wiley.com/doi/epdf/10.1029/2007GL032834

Hoshin V. Gupta

Papers

Bayesian Recursive Estimation of Parameter and Output Uncertainty for Watershed Models

Revisiting the Basis of Sensitivity Analysis for Dynamical Earth System Models

Improving robustness of hydrologic parameter estimation by the use of moving block bootstrap resampling

Estimating information entropy for hydrological data: One‐dimensional case

Toward a model space and model independence metric