Model Space Localization Is Not Always Better Than Observation Space Localization for Assimilation of Satellite Radiances
05 Oct 2015-Monthly Weather Review (American Meteorological Society)-Vol. 143, Iss: 10, pp 3948-3955
TL;DR: In this article, the detailed differences between model space and observation space localizations are examined using a real radiance observation, where the concepts of location or vertical distance are not well defined, vertical localization in observation space is not as straightforward as in model space.
Abstract: Covariance localization is an essential component of ensemble-based data assimilation systems for large geophysical applications with limited ensemble sizes. For integral observations like the satellite radiances, where the concepts of location or vertical distance are not well defined, vertical localization in observation space is not as straightforward as in model space. The detailed differences between model space and observation space localizations are examined using a real radiance observation. Counterintuitive analysis increments can be obtained with model space localization; the magnitude of the increment can increase and the increment can change sign when the localization scale decreases. This occurs when there are negative background-error covariances and a predominately positive forward operator. Too narrow model space localization can neglect the negative background-error covariances and result in the counterintuitive analysis increments. An idealized 1D model with integral observations...
Citations
More filters
TL;DR: For the EnKF in general, but higher-resolution applications in particular, it is desirable to use a short assimilation window, and a focus on approaches for maintaining balance during the EnkF update is focused on.
Abstract: This paper reviews the development of the ensemble Kalman filter (EnKF) for atmospheric data assimilation. Particular attention is devoted to recent advances and current challenges. The distinguishing properties of three well-established variations of the EnKF algorithm are first discussed. Given the limited size of the ensemble and the unavoidable existence of errors whose origin is unknown (i.e., system error), various approaches to localizing the impact of observations and to accounting for these errors have been proposed. However, challenges remain; for example, with regard to localization of multiscale phenomena (both in time and space). For the EnKF in general, but higher-resolution applications in particular, it is desirable to use a short assimilation window. This motivates a focus on approaches for maintaining balance during the EnKF update. Also discussed are limited-area EnKF systems, in particular with regard to the assimilation of radar data and applications to tracking severe storms ...
453 citations
TL;DR: In this article, the authors assimilated all-sky every-10-min infrared radiances from Himawari-8 with a regional numerical weather prediction model and investigated its impact on real-world tropical cyclone (TC) analyses and forecasts for the first time.
Abstract: Japan’s new geostationary satellite Himawari-8, the first of a series of the third-generation geostationary meteorological satellites including GOES-16, has been operational since July 2015. Himawari-8 produces high-resolution observations with 16 frequency bands every 10 min for full disk, and every 2.5 min for local regions. This study aims to assimilate all-sky every-10-min infrared (IR) radiances from Himawari-8 with a regional numerical weather prediction model and to investigate its impact on real-world tropical cyclone (TC) analyses and forecasts for the first time. The results show that the assimilation of Himawari-8 IR radiances improves the analyzed TC structure in both inner-core and outer-rainband regions. The TC intensity forecasts are also improved due to Himawari-8 data because of the improved TC structure analysis.
109 citations
TL;DR: In this paper, a modulation approach was adopted to implement model space localization in the operational National Oceanic and Atmospheric Administration EnKF, and the expanded ensemble is a square root of the vertically localized background error covariance matrix.
Abstract: Experiments using the National Oceanic and Atmospheric Administration Finite‐Volume Cubed‐Sphere Dynamical Core Global Forecasting System (FV3GFS) reveal that the four‐dimensional ensemble‐variational method (4DEnVAR) performs similarly to an ensemble Kalman filter (EnKF) when no radiance observations are assimilated, but 4DEnVAR is superior to an EnKF when radiance observations are assimilated. The hypothesis for the cause of the differences between 4DEnVAR and EnKF is the difference in vertical localization, since radiance observations are integral observations in the vertical and 4DEnVAR uses model space localization while the EnKF uses observation space localization. A modulation approach, which generates an expanded ensemble from the raw ensemble and eigenvectors of the localization matrix, has been adopted to implement model space localization in the operational National Oceanic and Atmospheric Administration EnKF. As constructed, the expanded ensemble is a square root of the vertically localized background error covariance matrix, so no explicit vertical localization is necessary during the EnKF update. The size of the expanded ensemble is proportional to the rank of the vertical localization matrix—for a vertical localization scale of 1.5 (3.0) scale heights, 12 (7) eigenvectors explain 96% of the variance of the localization matrix, so the expanded ensemble is 12 (7) times larger than the raw ensemble. Results from assimilating only radiance observations in the FV3GFS model confirm that EnKF with model‐space vertical localization performs better than observation‐space localization, and produces results similar to 4DEnVAR. Moreover, a 960‐member ensemble is sufficient to turn off the vertical localization entirely and yields significant improvements comparing to an 80‐member ensemble with model space localization.
42 citations
TL;DR: In this article, a variational data assimilation approach for both deterministic and ensemble prediction is proposed. But, the analysis increment is computed with a Variational Data Assimilation (VDA) approach for the ensemble mean and for all of the ensemble perturbations.
Abstract: Several NWP centers currently employ a variational data assimilation approach for initializing deterministic forecasts and a separate ensemble Kalman filter (EnKF) system both for initializing ensemble forecasts and for providing ensemble background error covariances for the deterministic system. This study describes a new approach for performing the data assimilation step within a perturbed-observation EnKF. In this approach, called VarEnKF, the analysis increment is computed with a variational data assimilation approach both for the ensemble mean and for all of the ensemble perturbations. To obtain a computationally efficient algorithm, a much simpler configuration is used for the ensemble perturbations, whereas the configuration used for the ensemble mean is similar to that used for the deterministic system. Numerous practical benefits may be realized by using a variational approach for both deterministic and ensemble prediction, including improved efficiency for the development and maintenance...
22 citations
References
More filters
TL;DR: In this article, a new sequential data assimilation method is proposed based on Monte Carlo methods, a better alternative than solving the traditional and computationally extremely demanding approximate error covariance equation used in the extended Kalman filter.
Abstract: A new sequential data assimilation method is discussed. It is based on forecasting the error statistics using Monte Carlo methods, a better alternative than solving the traditional and computationally extremely demanding approximate error covariance equation used in the extended Kalman filter. The unbounded error growth found in the extended Kalman filter, which is caused by an overly simplified closure in the error covariance equation, is completely eliminated. Open boundaries can be handled as long as the ocean model is well posed. Well-known numerical instabilities associated with the error covariance equation are avoided because storage and evolution of the error covariance matrix itself are not needed. The results are also better than what is provided by the extended Kalman filter since there is no closure problem and the quality of the forecast error statistics therefore improves. The method should be feasible also for more sophisticated primitive equation models. The computational load for reasonable accuracy is only a fraction of what is required for the extended Kalman filter and is given by the storage of, say, 100 model states for an ensemble size of 100 and thus CPU requirements of the order of the cost of 100 model integrations. The proposed method can therefore be used with realistic nonlinear ocean models on large domains on existing computers, and it is also well suited for parallel computers and clusters of workstations where each processor integrates a few members of the ensemble.
4,816 citations
TL;DR: In this article, the authors proposed an ensemble Kalman filter for data assimilation using the flow-dependent statistics calculated from an ensemble of short-range forecasts (a technique referred to as Ensemble Kalman filtering) in an idealized environment.
Abstract: The possibility of performing data assimilation using the flow-dependent statistics calculated from an ensemble of short-range forecasts (a technique referred to as ensemble Kalman filtering) is examined in an idealized environment. Using a three-level, quasigeostrophic, T21 model and simulated observations, experiments are performed in a perfect-model context. By using forward interpolation operators from the model state to the observations, the ensemble Kalman filter is able to utilize nonconventional observations. In order to maintain a representative spread between the ensemble members and avoid a problem of inbreeding, a pair of ensemble Kalman filters is configured so that the assimilation of data using one ensemble of shortrange forecasts as background fields employs the weights calculated from the other ensemble of short-range forecasts. This configuration is found to work well: the spread between the ensemble members resembles the difference between the ensemble mean and the true state, except in the case of the smallest ensembles. A series of 30-day data assimilation cycles is performed using ensembles of different sizes. The results indicate that (i) as the size of the ensembles increases, correlations are estimated more accurately and the root-meansquare analysis error decreases, as expected, and (ii) ensembles having on the order of 100 members are sufficient to accurately describe local anisotropic, baroclinic correlation structures. Due to the difficulty of accurately estimating the small correlations associated with remote observations, a cutoff radius beyond which observations are not used, is implemented. It is found that (a) for a given ensemble size there is an optimal value of this cutoff radius, and (b) the optimal cutoff radius increases as the ensemble size increases.
1,827 citations
TL;DR: In this article, it is shown that the observations must be treated as random variables at the analysis steps, which results in a completely consistent approach if the covariance of the ensemble of model states is interpreted as the prediction error covariance, and there are no further requirements on the ensemble Kalman filter method.
Abstract: This paper discusses an important issue related to the implementation and interpretation of the analysis scheme in the ensemble Kalman filter. It is shown that the observations must be treated as random variables at the analysis steps. That is, one should add random perturbations with the correct statistics to the observations and generate an ensemble of observations that then is used in updating the ensemble of model states. Traditionally, this has not been done in previous applications of the ensemble Kalman filter and, as will be shown, this has resulted in an updated ensemble with a variance that is too low. This simple modification of the analysis scheme results in a completely consistent approach if the covariance of the ensemble of model states is interpreted as the prediction error covariance, and there are no further requirements on the ensemble Kalman filter method, except for the use of an ensemble of sufficient size. Thus, there is a unique correspondence between the error statistics from the ensemble Kalman filter and the standard Kalman filter approach.
1,801 citations
TL;DR: In this paper, the authors focus on the construction of simply parametrized covariance functions for data-assimilation applications and provide a self-contained, rigorous mathematical summary of relevant topics from correlation theory.
Abstract: This article focuses on the construction, directly in physical space, of simply parametrized covariance functions for data-assimilation applications. A self-contained, rigorous mathematical summary of relevant topics from correlation theory is provided as a foundation for this construction. Covariance and correlation functions are defined, and common notions of homogeneity and isotropy are clarified. Classical results are stated, and proven where instructive. Included are smoothness properties relevant to multivariate statistical-analysis algorithms where wind/wind and wind/mass correlation models are obtained by differentiating the correlation model of a mass variable. the Convolution Theorem is introduced as the primary tool used to construct classes of covariance and cross-covariance functions on three-dimensional Euclidean space R3. Among these are classes of compactly supported functions that restrict to covariance and cross-covariance functions on the unit sphere S2, and that vanish identically on subsets of positive measure on S2. It is shown that these covariance and cross-covariance functions on S2, referred to as being space-limited, cannot be obtained using truncated spectral expansions. Compactly supported and space-limited covariance functions determine sparse covariance matrices when evaluated on a grid, thereby easing computational burdens in atmospheric data-analysis algorithms.
Convolution integrals leading to practical examples of compactly supported covariance and cross-covariance functions on R3 are reduced and evaluated. More specifically, suppose that gi and gj are radially symmetric functions defined on R3 such that gi(x) = 0 for |x| > di and gj(x) = 0 for |xv > dj, O di + dj and |x - y| > 2di, respectively, Additional covariance functions on R3 are constructed using convolutions over the real numbers R, rather than R3. Families of compactly supported approximants to standard second- and third-order autoregressive functions are constructed as illustrative examples. Compactly supported covariance functions of the form C(x,y) := Co(|x - y|), x,y ∈ R3, where the functions Co(r) for r ∈ R are 5th-order piecewise rational functions, are also constructed. These functions are used to develop space-limited product covariance functions B(x, y) C(x, y), x, y ∈ S2, approximating given covariance functions B(x, y) supported on all of S2 × S2.
1,770 citations
TL;DR: In this article, an ensemble Kalman filter is proposed for the 4D assimilation of atmospheric data, which employs a Schur (elementwise) product of the covariances of the background error calculated from the ensemble and a correlation function having local support to filter the small (and noisy) background-error covariance associated with remote observations.
Abstract: An ensemble Kalman filter may be considered for the 4D assimilation of atmospheric data. In this paper, an efficient implementation of the analysis step of the filter is proposed. It employs a Schur (elementwise) product of the covariances of the background error calculated from the ensemble and a correlation function having local support to filter the small (and noisy) background-error covariances associated with remote observations. To solve the Kalman filter equations, the observations are organized into batches that are assimilated sequentially. For each batch, a Cholesky decomposition method is used to solve the system of linear equations. The ensemble of background fields is updated at each step of the sequential algorithm and, as more and more batches of observations are assimilated, evolves to eventually become the ensemble of analysis fields. A prototype sequential filter has been developed. Experiments are performed with a simulated observational network consisting of 542 radiosonde and 615 satellite-thickness profiles. Experimental results indicate that the quality of the analysis is almost independent of the number of batches (except when the ensemble is very small). This supports the use of a sequential algorithm. A parallel version of the algorithm is described and used to assimilate over 100 000 observations into a pair of 50-member ensembles. Its operation count is proportional to the number of observations, the number of analysis grid points, and the number of ensemble members. In view of the flexibility of the sequential filter and its encouraging performance on a NEC SX-4 computer, an application with a primitive equations model can now be envisioned.
1,444 citations