scispace - formally typeset
Search or ask a question
Journal ArticleDOI

A Comparison of Precipitation Forecast Skill between Small Convection-Allowing and Large Convection-Parameterizing Ensembles

01 Aug 2009-Weather and Forecasting (American Meteorological Society)-Vol. 24, Iss: 4, pp 1121-1140
TL;DR: In this paper, an experiment was designed to evaluate and compare precipitation forecasts from a 5-member, 4-km grid-spacing (ENS4) and a 15-members, 20-km Grid-Spaces (ENS20) model ensemble, which cover a similar domain over the central United States.
Abstract: An experiment has been designed to evaluate and compare precipitation forecasts from a 5-member, 4-km grid-spacing (ENS4) and a 15-member, 20-km grid-spacing (ENS20) Weather Research and Forecasting (WRF) model ensemble, which cover a similar domain over the central United States. The ensemble forecasts are initialized at 2100 UTC on 23 different dates and cover forecast lead times up to 33 h. Previous work has demonstrated that simulations using convection-allowing resolution (CAR; dx ; 4 km) have a better representation of the spatial and temporal statistical properties of convective precipitation than coarser models using convective parameterizations. In addition, higher resolution should lead to greater ensemble spread as smaller scales of motion are resolved. Thus, CAR ensembles should provide more accurate and reliable probabilistic forecasts than parameterized-convection resolution (PCR) ensembles. Computation of various precipitation skill metrics for probabilistic and deterministic forecasts reveals that ENS4 generally provides more accurate precipitation forecasts than ENS20, with the differences tending to be statistically significant for precipitation thresholds above 0.25 in. at forecast lead times of 9‐21 h (0600‐ 1800 UTC) for all accumulation intervals analyzed (1, 3, and 6 h). In addition, an analysis of rank histograms and statistical consistency reveals that faster error growth in ENS4 eventually leads to more reliable precipitation forecasts in ENS4 than in ENS20. For the cases examined, these results imply that the skill gained by increasing to CAR outweighs the skill lost by decreasing the ensemble size. Thus, when computational capabilities become available, it will be highly desirable to increase the ensemble resolution from PCR to CAR, even if the size of the ensemble has to be reduced.

Summary (2 min read)

Introduction

  • An experiment is designed to evaluate and compare precipitation forecasts from a 5-member, 4-km grid-spacing (ENS4) and a 15-member, 20-km grid-spacing (ENS20) Weather Research and Forecasting (WRF) model ensemble, which cover a similar domain over the central United States.
  • Several factors are probably contributing to this lack of spread including coarsely resolved and temporally interpolated lateral boundary conditions (LBCs; Nutter et al. 2003), inappropriate IC perturbation strategies for short-ranges (Eckel and Mass 2005), and inability to capture small-scale variability because of insufficient resolution (Eckel and Mass 2005).
  • Because of these limitations, significant improvements in rainfall forecasts may be realized by running an ensemble using explicit representation of convection (i.e., no CP).
  • So, the purpose of this study is not to compare ensembles with similar computational expense, but to determine if at some point when computational capabilities allow, it would be advantageous to reduce ensemble size in order to use CAR.

2. Ensemble descriptions and cases examined

  • The ENS4 ensemble was obtained from a real-time ensemble forecasting experiment conducted as part of the NOAA Hazardous Weather Testbed (HWT) Spring Experiment (Kain et al. 2008) during April-June 2007 (Xue et al. 2007; Kong et al. 2007).
  • For the initial perturbed members, perturbations extracted from the 2100 UTC NCEP SREF WRF-ARW and WRF-NMM members are added to the 2100 UTC NAM analyses, and the corresponding SREF forecasts are used for LBCs (3-hr updates).
  • The ENS20 ensemble was generated at Iowa State University and is also composed of WRF- ARW (Version 2.2.0) members with perturbed ICs/LBCs and mixed-physics.
  • ENS4 and ENS20 specifications are listed in Tables 1 and 2, respectively.
  • These cases were chosen based on the availability of the ENS4 real-time forecasts and represent a variety of convective precipitation events [e.g., isolated convection (4/19), heavy rainfall associated with a cutoff upper-low (4/22 – 4/25), and many nocturnal MCSs (late May/early June)].

3. Data and Methodology

  • This study examines forecasts of 1- 3- and 6-hrly accumulated rainfall.
  • The ENS4 forecasts were coarsened to allow direct comparisons to ENS20; additional and potentially useful information on finer-scale details in the forecast precipitation fields could be gained using the ENS4 data on its original 4-km grid.
  • The ensemble mean obtained from the probability matching procedure (PM hereafter) can help correct for large biases in areal rainfall coverage and underestimation of rainfall amounts that are typically associated with using a standard ensemble mean, and results in a precipitation field with a much more realistic distribution.
  • To verify probabilistic forecasts, the area under the relative operating characteristic curve (ROC score; Mason 1982) is used, which is closely related to the economic value of a forecast system (e.g., Mylne 1999; Richardson 2000, 2001).
  • Because the method used to compute FPs in this study allows for continuous (rather than discrete) values of FPs between 0 and 100%, the same set of FP ranges that make up the points on the ROC curve can be used to verify both ensembles, and problems associated with comparing ROC scores between ensembles of different sizes are avoided.

4. Results

  • A. Analysis of diurnally-averaged diagrams Warm season precipitation in the central United States tends to form at similar times of day and propagate over similar longitudes so that when diurnally averaged time-longitude diagrams of precipitation are constructed, coherent and propagating rainfall axes are observed (Carbone et al. 2002).
  • These results are a strong indication that the ability of the CAR members in ENS4 to properly simulate propagating MCSs explains the statistically significant differences in ETS between ENS4 and ENS20 observed in Figure 4. c. Comparison of ROC scores.
  • There also appears to be a secondary maximum in ENS4 ROC scores at the 0.50- and 1.00-in rainfall thresholds for all accumulation intervals examined around forecast hour 27 (Figs. 7d-i).
  • To allow for a more convenient comparison, the 16 bins composing the ENS20 rank histogram are regrouped into 6 bins which each contain an equal portion of the original 16 bins (Fig. 8a).
  • At all forecast lead times, the right-skewness of rank histograms from both ensembles indicates a tendency for members to over-predict precipitation (Figs. 8a, b).

2) STATISTICAL CONSISTENCY ANALYSIS

  • Ensembles correctly forecasting uncertainty are considered statistically consistent, and the mean-square-error (mse) of the ensemble mean will match the ensemble variance when averaged over many cases (Talagrand et al.
  • The discrepancy between rank histogram and statistical consistency results (Figs. 10a, c, and e) highlights the importance of recognizing the effects of bias when interpreting statistical consistency analyses.
  • When differences in ensemble variance were similar at forecast hour 18 (Fig. 11a), biases at rainfall thresholds above 0.25-in were also similar (Fig. 11b).
  • First, note that the faster error growth inferred from ensemble variance in ENS4 relative to ENS20 up to forecast hour 9 (at 1- and 3-hrly accumulation intervals), and after forecast hour 21 (all accumulation intervals; Figs. 10a, c, and e), is largely an artifact of bias.

Acknowledgments

  • The authors would like to thank Huiling Yuan at the Global Systems Division of the Earth System Research Laboratory (ESRL GSD) for assistance in obtaining SREF data in post real-time.
  • This particular research was funded by NSF Grant ATM-0537043.
  • Kevin Thomas carried out the real-time runs.
  • The CAPS real-time predictions were performed at the Pittsburgh Supercomputing Center (PSC) supported by NSF.
  • Finally, constructive comments from three reviewers were appreciated.

Did you find this useful? Give us your feedback

Figures (14)

Content maybe subject to copyright    Report

Citations
More filters
Journal ArticleDOI
TL;DR: The extent to which the current ECMWF ensemble prediction system is capable of predicting flow-dependent variations in uncertainty is assessed for the large-scale flow in mid-latitudes.

738 citations

Journal ArticleDOI
TL;DR: In this paper, the authors summarized the recent progress and discussed some of the challenges for future advancement in the use of high-resolution numerical weather prediction (NWP) for nowcasting.
Abstract: Traditionally, the nowcasting of precipitation was conducted to a large extent by means of extrapolation of observations, especially of radar ref lectivity. In recent years, the blending of traditional extrapolation-based techniques with high-resolution numerical weather prediction (NWP) is gaining popularity in the nowcasting community. The increased need of NWP products in nowcasting applications poses great challenges to the NWP community because the nowcasting application of high-resolution NWP has higher requirements on the quality and content of the initial conditions compared to longer-range NWP. Considerable progress has been made in the use of NWP for nowcasting thanks to the increase in computational resources, advancement of high-resolution data assimilation techniques, and improvement of convective-permitting numerical modeling. This paper summarizes the recent progress and discusses some of the challenges for future advancement.

341 citations

Journal ArticleDOI
TL;DR: In the 2007 NOAA Hazardous Weather Testbed Spring Experiment, the Center for Analysis and Prediction of Storms (CAPS) at the University of Oklahoma produced a daily 10-member 4-km horizontal resolution ensemble forecast covering approximately three-fourths of the continental United States as mentioned in this paper.
Abstract: During the 2007 NOAA Hazardous Weather Testbed Spring Experiment, the Center for Analysis and Prediction of Storms (CAPS) at the University of Oklahoma produced a daily 10-member 4-km horizontal resolution ensemble forecast covering approximately three-fourths of the continental United States. Each member used the Advanced Research version of the Weather Research and Forecasting (WRF-ARW) model core, which was initialized at 2100 UTC, ran for 33 h, and resolved convection explicitly. Different initial condition (IC), lateral boundary condition (LBC), and physics perturbations were introduced in 4 of the 10 ensemble members, while the remaining 6 members used identical ICs and LBCs, differing only in terms of microphysics (MP) and planetary boundary layer (PBL) parameterizations. This study focuses on precipitation forecasts from the ensemble. The ensemble forecasts reveal WRF-ARW sensitivity to MP and PBL schemes. For example, over the 7-week experiment, the Mellor–Yamada–Janjic PBL and Ferrier M...

224 citations

Journal ArticleDOI
TL;DR: The 2010 NOAA/HWT Spring Forecasting Experiment (SE2010), conducted 17 May through 18 June, had a broad focus, with emphases on heavy rainfall and aviation weather, through collaboration with the Hydrometeorological Prediction Center (HPC) and the Aviation Weather Center (AWC), respectively as discussed by the authors.
Abstract: The NOAA Hazardous Weather Testbed (HWT) conducts annual spring forecasting experiments organized by the Storm Prediction Center and National Severe Storms Laboratory to test and evaluate emerging scientific concepts and technologies for improved analysis and prediction of hazardous mesoscale weather. A primary goal is to accelerate the transfer of promising new scientific concepts and tools from research to operations through the use of intensive real-time experimental forecasting and evaluation activities conducted during the spring and early summer convective storm period. The 2010 NOAA/HWT Spring Forecasting Experiment (SE2010), conducted 17 May through 18 June, had a broad focus, with emphases on heavy rainfall and aviation weather, through collaboration with the Hydrometeorological Prediction Center (HPC) and the Aviation Weather Center (AWC), respectively. In addition, using the computing resources of the National Institute for Computational Sciences at the University of Tennessee, the Center for A...

220 citations

Journal ArticleDOI
TL;DR: In this paper, the effects of grid-size dependency in the vertical heat transport parameterization for convective boundary layers (CBLs) were investigated by investigating the effect of grid size dependency on the vertical transport profile.
Abstract: Parameterization of the unresolved vertical transport in the planetary boundary layer (PBL) is one of the key physics algorithms in atmospheric models. This study attempts to represent the subgrid-scale (SGS) turbulent transport in convective boundary layers (CBLs) at gray-zone resolutions by investigating the effects of grid-size dependency in the vertical heat transport parameterization for CBL simulations. The SGS transport profile is parameterized based on the 2013 conceptual derivation by Shin and Hong. First, nonlocal transport via strong updrafts and local transport via the remaining small-scale eddies are separately calculated. Second, the SGS nonlocal transport is formulated by multiplying a grid-size dependency function with the total nonlocal transport profile fit to the large-eddy simulation (LES) output. Finally, the SGS local transport is formulated by multiplying a grid-size dependency function with the total local transport profile, which is calculated using an eddy-diffusivity for...

213 citations

References
More filters
DOI
01 Jan 2008
TL;DR: The Technical Note series provides an outlet for a variety of NCAR manuscripts that contribute in specialized ways to the body of scientific knowledge but which are not suitable for journal, monograph, or book publication.
Abstract: The Technical Note series provides an outlet for a variety of NCAR manuscripts that contribute in specialized ways to the body of scientific knowledge but which are not suitable for journal, monograph, or book publication. Reports in this series are issued by the NCAR Scientific Divisions ; copies may be obtained on request from the Publications Office of NCAR. Designation symbols for the series include: Any opinions, findings, conclusions, or recommendations expressed in this publication are those of the authors and do not necessarily reflect the views of the National Science Foundation.

9,022 citations

Journal ArticleDOI
TL;DR: A rapid and accurate radiative transfer model (RRTM) for climate applications has been developed and the results extensively evaluated as discussed by the authors, which is performed using the correlated-k method: the k distributions are attained directly from the LBLRTM line-byline model, which connects the absorption coefficients used by RRTM to high-resolution radiance validations done with observations.
Abstract: A rapid and accurate radiative transfer model (RRTM) for climate applications has been developed and the results extensively evaluated. The current version of RRTM calculates fluxes and cooling rates for the longwave spectral region (10–3000 cm−1) for an arbitrary clear atmosphere. The molecular species treated in the model are water vapor, carbon dioxide, ozone, methane, nitrous oxide, and the common halocarbons. The radiative transfer in RRTM is performed using the correlated-k method: the k distributions are attained directly from the LBLRTM line-by-line model, which connects the absorption coefficients used by RRTM to high-resolution radiance validations done with observations. Refined methods have been developed for treating bands containing gases with overlapping absorption, for the determination of values of the Planck function appropriate for use in the correlated-k approach, and for the inclusion of minor absorbing species in a band. The flux and cooling rate results of RRTM are linked to measurement through the use of LBLRTM, which has been substantially validated with observations. Validations of RRTM using LBLRTM have been performed for the midlatitude summer, tropical, midlatitude winter, subarctic winter, and four atmospheres from the Spectral Radiance Experiment campaign. On the basis of these validations the longwave accuracy of RRTM for any atmosphere is as follows: 0.6 W m−2 (relative to LBLRTM) for net flux in each band at all altitudes, with a total (10–3000 cm−1) error of less than 1.0 W m−2 at any altitude; 0.07 K d−1 for total cooling rate error in the troposphere and lower stratosphere, and 0.75 K d−1 in the upper stratosphere and above. Other comparisons have been performed on RRTM using LBLRTM to gauge its sensitivity to changes in the abundance of specific species, including the halocarbons and carbon dioxide. The radiative forcing due to doubling the concentration of carbon dioxide is attained with an accuracy of 0.24 W m−2, an error of less than 5%. The speed of execution of RRTM compares favorably with that of other rapid radiation models, indicating that the model is suitable for use in general circulation models.

6,861 citations

Journal ArticleDOI
TL;DR: The second-moment turbulent closure hypothesis has been applied to geophysical fluid problems since 1973, when genuine predictive skill in coping with the effects of stratification was demonstrated as discussed by the authors.
Abstract: Applications of second-moment turbulent closure hypotheses to geophysical fluid problems have developed rapidly since 1973, when genuine predictive skill in coping with the effects of stratification was demonstrated. The purpose here is to synthesize and organize material that has appeared in a number of articles and add new useful material so that a complete (and improved) description of a turbulence model from conception to application is condensed in a single article. It is hoped that this will be a useful reference to users of the model for application to either atmospheric or oceanic boundary layers.

6,488 citations

25 Feb 2004
TL;DR: The instructor's manual to a work which introduces the fundamental principles of meteorology, explaining storm dynamics and the dynamics of climate and its global implications is described in this paper, where the authors present a detailed discussion of the relationship between meteorology and climate.
Abstract: The instructor's manual to a work which introduces the fundamental principles of meteorology, explaining storm dynamics and the dynamics of climate and its global implications.

4,185 citations

Journal ArticleDOI
TL;DR: In this article, a major revision of the Betts and Miller convection scheme was made, a new marine viscous sublayer scheme was designed, and the Mellor-Yamada planetary boundary layer (PBL) formulation was retuned.
Abstract: The step-mountain eta model has shown a surprising skill in forecasting severe storms. Much of the credit for this should be given to the Betts and Miller (hereafter referred to as BM) convection scheme and the Mellor-Yamada (hereafter referred to as MY) planetary boundary layer (PBL) formulation. However, the eta model was occasionally producing heavy spurious precipitation over warm water, as well as widely spread light precipitation over oceans. In addition, the convective forcing, particularly the shallow one, could lead to negative entropy changes. As the possible causes of the problems, the convection scheme, the processes at the air-water interface, and the MY level 2 and level 2.5 PBL schemes were reexamined. A major revision of the BM scheme was made, a new marine viscous sublayer scheme was designed, and the MY schemes were retuned. The deep convective regimes are postulated to be characterized by a parameter called “cloud efficiency.” The relaxation time is extended for low cloud effic...

2,836 citations

Frequently Asked Questions (1)
Q1. What are the future works mentioned in the paper "A comparison of precipitation forecast skill between small convection- allowing and large convection-parameterizing ensembles" ?

Future work should explore CAR ensembles using more members and larger sets of cases for convective and non-convective precipitation events.