Watershed models are powerful tools for simulating the effect of watershed processes and management on soil and water resources. However, no comprehensive guidance is available to facilitate model evaluation in terms of the accuracy of simulated data compared to measured flow and constituent values. Thus, the objectives of this research were to: (1) determine recommended model evaluation techniques (statistical and graphical), (2) review reported ranges of values and corresponding performance ratings for the recommended statistics, and (3) establish guidelines for model evaluation based on the review results and project-specific considerations; all of these objectives focus on simulation of streamflow and transport of sediment and nutrients. These objectives were achieved with a thorough review of relevant literature on model application and recommended model evaluation methods. Based on this analysis, we recommend that three quantitative statistics, Nash-Sutcliffe efficiency (NSE), percent bias (PBIAS), and ratio of the root mean square error to the standard deviation of measured data (RSR), in addition to the graphical techniques, be used in model evaluation. The following model evaluation performance ratings were established for each recommended statistic. In general, model simulation can be judged as satisfactory if NSE > 0.50 and RSR < 0.70, and if PBIAS + 25% for streamflow, PBIAS + 55% for sediment, and PBIAS + 70% for N and P. For PBIAS, constituent-specific performance ratings were determined based on uncertainty of measured data. Additional considerations related to model evaluation guidelines are also discussed. These considerations include: single-event simulation, quality and quantity of measured data, model calibration procedure, evaluation time step, and project scope and magnitude. A case study illustrating the application of the model evaluation guidelines is also provided.

Model Evaluation Guidelines for Systematic Quantification of Accuracy in Watershed Simulations

Statistics for Spatial Data.

The Tropical Rainfall Measuring Mission (TRMM) Multisatellite Precipitation Analysis (TMPA) provides a calibration-based sequential scheme for combining precipitation estimates from multiple satellites, as well as gauge analyses where feasible, at fine scales (0.25° × 0.25° and 3 hourly). TMPA is available both after and in real time, based on calibration by the TRMM Combined Instrument and TRMM Microwave Imager precipitation products, respectively. Only the after-real-time product incorporates gauge data at the present. The dataset covers the latitude band 50°N–S for the period from 1998 to the delayed present. Early validation results are as follows: the TMPA provides reasonable performance at monthly scales, although it is shown to have precipitation rate–dependent low bias due to lack of sensitivity to low precipitation rates over ocean in one of the input products [based on Advanced Microwave Sounding Unit-B (AMSU-B)]. At finer scales the TMPA is successful at approximately reproducing the s...

/pdf/the-trmm-multisatellite-precipitation-analysis-tmpa-quasi-3wulgfnahi.pdf

The TRMM Multisatellite Precipitation Analysis (TMPA): Quasi-Global, Multiyear, Combined-Sensor Precipitation Estimates at Fine Scales

List of Figures. List of Tables. Preface. Foreword. 1. Basic Concepts. 2. Evolutionary Algorithm MOP Approaches. 3. MOEA Test Suites. 4. MOEA Testing and Analysis. 5. MOEA Theory and Issues. 3. MOEA Theoretical Issues. 6. Applications. 7. MOEA Parallelization. 8. Multi-Criteria Decision Making. 9. Special Topics. 10. Epilog. Appendix A: MOEA Classification and Technique Analysis. Appendix B: MOPs in the Literature. Appendix C: Ptrue & PFtrue for Selected Numeric MOPs. Appendix D: Ptrue & PFtrue for Side-Constrained MOPs. Appendix E: MOEA Software Availability. Appendix F: MOEA-Related Information. Index. References.

/pdf/evolutionary-algorithms-for-solving-multi-objective-problems-2txxk4mqci.pdf

Evolutionary algorithms for solving multi-objective problems

Decomposition of the mean squared error and NSE performance criteria: Implications for improving hydrological modelling

Diagnostic Evaluation of a Distributed Precipitation-Runoff Model for Snow Dominated Basins

https://agupubs.onlinelibrary.wiley.com/doi/epdf/10.1029/2019WR025605

Improving information extraction from simulated discharge using sensitivity‐weighted performance criteria

Toward Improved Calibration of a Semi-arid Distributed Flash-Flood Model: A Hierarchical Sensitivity Scheme for Model Evaluation

We develop a simple Quantile Spacing (QS) method for accurate probabilistic estimation of one-dimensional entropy from equiprobable random samples, and compare it with the popular Bin-Counting (BC) and Kernel Density (KD) methods. In contrast to BC, which uses equal-width bins with varying probability mass, the QS method uses estimates of the quantiles that divide the support of the data generating probability density function (pdf) into equal-probability-mass intervals. And, whereas BC and KD each require optimal tuning of a hyper-parameter whose value varies with sample size and shape of the pdf, QS only requires specification of the number of quantiles to be used. Results indicate, for the class of distributions tested, that the optimal number of quantiles is a fixed fraction of the sample size (empirically determined to be ~0.25–0.35), and that this value is relatively insensitive to distributional form or sample size. This provides a clear advantage over BC and KD since hyper-parameter tuning is not required. Further, unlike KD, there is no need to select an appropriate kernel-type, and so QS is applicable to pdfs of arbitrary shape, including those with discontinuous slope and/or magnitude. Bootstrapping is used to approximate the sampling variability distribution of the resulting entropy estimate, and is shown to accurately reflect the true uncertainty. For the four distributional forms studied (Gaussian, Log-Normal, Exponential and Bimodal Gaussian Mixture), expected estimation bias is less than 1% and uncertainty is low even for samples of as few as 100 data points; in contrast, for KD the small sample bias can be as large as −10% and for BC as large as −50%. We speculate that estimating quantile locations, rather than bin-probabilities, results in more efficient use of the information in the data to approximate the underlying shape of an unknown data generating pdf.

/pdf/computing-accurate-probabilistic-estimates-of-one-d-entropy-2i367hxtt5.pdf

Computing accurate probabilistic estimates of one‐d entropy from equiprobable random samples

This paper evaluates two contrasting approaches to parameter estimation for ungauged basins using the US National Weather Service's SACramento Soil Moisture Accounting (SAC-SMA) model. An automatic calibration scheme (Multi-Step Automatic Calibration Scheme, MACS) provides deterministic parameter estimates using a three-step, multiple objective approach. The MACS estimates are then transferred to similar or sister watersheds for basins in the French MOPEX data set. Physically-based parameter estimates are also developed for the same basins based on the a priori approach of Koren et al. (2000). In general, the two methods, the transfer and the a priori approaches, show similar overall performance. Parameter estimates appear more consistent between basins using the a priori approach, but statistically the regionalized MACS parameters and the a priori parameters show very similar model performance for the three basins investigated in this study. Model simulated hydrographs are also very similar between the two methods, with both methods tending to underpredict most events (peak and volume) but matching the shape and pattern of flow well. However, both methods have worse performance than a calibrated model for the same basin, indicating the possibility for further refinement and adjustment of the techniques presented here.

Hoshin V. Gupta

Papers

Diagnostic Evaluation of a Distributed Precipitation-Runoff Model for Snow Dominated Basins

Improving information extraction from simulated discharge using sensitivity‐weighted performance criteria

Toward Improved Calibration of a Semi-arid Distributed Flash-Flood Model: A Hierarchical Sensitivity Scheme for Model Evaluation

Computing accurate probabilistic estimates of one‐d entropy from equiprobable random samples

Modelling ungauged basins with the Sacramento model