The purpose of this paper is to provide a comprehensive presentation and interpretation of the Ensemble Kalman Filter (EnKF) and its numerical implementation. The EnKF has a large user group, and numerous publications have discussed applications and theoretical aspects of it. This paper reviews the important results from these studies and also presents new ideas and alternative interpretations which further explain the success of the EnKF. In addition to providing the theoretical framework needed for using the EnKF, there is also a focus on the algorithmic formulation and optimal numerical implementation. A program listing is given for some of the key subroutines. The paper also touches upon specific issues such as the use of nonlinear measurements, in situ profiles of temperature and salinity, and data which are available with high frequency in time. An ensemble based optimal interpolation (EnOI) scheme is presented as a cost-effective approach which may serve as an alternative to the EnKF in some applications. A fairly extensive discussion is devoted to the use of time correlated model errors and the estimation of model bias.

/pdf/the-ensemble-kalman-filter-theoretical-formulation-and-2xx6out3z9.pdf

The Ensemble Kalman Filter: theoretical formulation and practical implementation

The Ensemble Kalman Filter: Theoretical formulation and practical implementation

Soil moisture controls the partitioning of moisture and energy fluxes at the land surface and is a key variable in weather and climate prediction. The performance of the ensemble Kalman filter (EnKF) for soil moisture estimation is assessed by assimilating L-band (1.4 GHz) microwave radiobrightness observations into a land surface model. An optimal smoother (a dynamic variational method) is used as a benchmark for evaluating the filter’s performance. In a series of synthetic experiments the effect of ensemble size and non-Gaussian forecast errors on the estimation accuracy of the EnKF is investigated. With a state vector dimension of 4608 and a relatively small ensemble size of 30 (or 100; or 500), the actual errors in surface soil moisture at the final update time are reduced by 55% (or 70%; or 80%) from the value obtained without assimilation (as compared to 84% for the optimal smoother). For robust error variance estimates, an ensemble of at least 500 members is needed. The dynamic evolution of the estimation error variances is dominated by wetting and drying events with high variances during drydown and low variances when the soil is either very wet or very dry. Furthermore, the ensemble distribution of soil moisture is typically symmetric except under very dry or wet conditions when the effects of the nonlinearities in the model become significant. As a result, the actual errors are consistently larger than ensemble-derived forecast and analysis error variances. This suggests that the update is suboptimal. However, the degree of suboptimality is relatively small and results presented here indicate that the EnKF is a flexible and robust data assimilation option that gives satisfactory estimates even for moderate ensemble sizes.

/pdf/hydrologic-data-assimilation-with-the-ensemble-kalman-filter-58bvoa3xe1.pdf

Hydrologic Data Assimilation with the Ensemble Kalman Filter

[1] Hydrologic models use relatively simple mathematical equations to conceptualize and aggregate the complex, spatially distributed, and highly interrelated water, energy, and vegetation processes in a watershed. A consequence of process aggregation is that the model parameters often do not represent directly measurable entities and must therefore be estimated using measurements of the system inputs and outputs. During this process, known as model calibration, the parameters are adjusted so that the behavior of the model approximates, as closely and consistently as possible, the observed response of the hydrologic system over some historical period of time. In practice, however, because of errors in the model structure and the input (forcing) and output data, this has proven to be difficult, leading to considerable uncertainty in the model predictions. This paper surveys the limitations of current model calibration methodologies, which treat the uncertainty in the input-output relationship as being primarily attributable to uncertainty in the parameters and presents a simultaneous optimization and data assimilation (SODA) method, which improves the treatment of uncertainty in hydrologic modeling. The usefulness and applicability of SODA is demonstrated by means of a pilot study using data from the Leaf River watershed in Mississippi and a simple hydrologic model with typical conceptual components.

https://agupubs.onlinelibrary.wiley.com/doi/pdf/10.1029/2004WR003059

Improved treatment of uncertainty in hydrologic modeling: Combining the strengths of global optimization and data assimilation

[1] The ability to monitor floods with sensors mounted on aircraft and satellites has been known for decades. Early launches of satellites and the availability of aerial photography allowed investigation of the potential to support flood monitoring from as far as space. There have been notable studies on integrating data from these instruments with flood modeling since the late 1990s. There is now a consensus among space agencies to strengthen the support that satellites can offer. This trend has stimulated more research in this area, and significant progress has been achieved in recent years in fostering our understanding of the ways in which remote sensing can support or even advance flood modeling. This research goes considerably further than using a wet/dry flood map for model validation as in early studies of this type. Therefore, this paper aims to review recent and current efforts to aid advancing flood inundation modeling from space.

Progress in integration of remote sensing– derived flood extent and stage data and hydraulic models

Data assimilation in a two-dimensional hydrodynamic model for bays, estuaries and coastal areas is considered. Two different methods based on the Kalman filter scheme are presented. These include (1) an extended Kalman filter in which the error covariance matrix is approximated by a matrix of reduced rank using a square root factorisation (RRSQRT KF), and (2) an ensemble Kalman filter (EnKF) based on a Monte Carlo simulation approach for propagation of errors. The filtering problem is formulated by utilising a general description of the model noise process related to errors in the model forcing, i.e. open boundary conditions and meteorological forcing. The performance of the two Kalman filters is evaluated using a twin experiment based on a hypothetical bay region. For both filters, the error covariance approximation sufficiently resolves the error propagation in the model at a computational load that is significantly smaller than required by the full Kalman filter algorithm. Furthermore, the Kalman filters are shown to be very robust with respect to defining the errors. Even in the case of a severely biased model error, the filters are able to efficiently correct the model. In general, the use of coloured model noise provides a numerically more efficient algorithm as well as a better performance of the filter. The error covariance approximation in the RRSQRT KF is shown to be more efficient than the error representation in the EnKF. For strongly non-linear dynamics, however, the EnKF is preferable. Copyright © 1999 John Wiley & Sons, Ltd.

/pdf/comparison-of-extended-and-ensemble-kalman-filters-for-data-4j8lj6u8dh.pdf

H. Madsen

Papers

Comparison of extended and ensemble Kalman filters for data assimilation in coastal area modelling