The purpose of this paper is to provide a comprehensive presentation and interpretation of the Ensemble Kalman Filter (EnKF) and its numerical implementation. The EnKF has a large user group, and numerous publications have discussed applications and theoretical aspects of it. This paper reviews the important results from these studies and also presents new ideas and alternative interpretations which further explain the success of the EnKF. In addition to providing the theoretical framework needed for using the EnKF, there is also a focus on the algorithmic formulation and optimal numerical implementation. A program listing is given for some of the key subroutines. The paper also touches upon specific issues such as the use of nonlinear measurements, in situ profiles of temperature and salinity, and data which are available with high frequency in time. An ensemble based optimal interpolation (EnOI) scheme is presented as a cost-effective approach which may serve as an alternative to the EnKF in some applications. A fairly extensive discussion is devoted to the use of time correlated model errors and the estimation of model bias.

The Ensemble Kalman Filter: Theoretical formulation and practical implementation

The growing need of the driving public for accurate traffic information has spurred the deployment of large scale dedicated monitoring infrastructure systems, which mainly consist in the use of inductive loop detectors and video cameras. On-board electronic devices have been proposed as an alternative traffic sensing infrastructure, as they usually provide a cost-effective way to collect traffic data, leveraging existing communication infrastructure such as the cellular phone network. A traffic monitoring system based on GPS-enabled smartphones exploits the extensive coverage provided by the cellular network, the high accuracy in position and velocity measurements provided by GPS devices, and the existing infrastructure of the communication network. This article presents a field experiment nicknamed Mobile Century, which was conceived as a proof of concept of such a system. Mobile Century included 100 vehicles carrying a GPS-enabled Nokia N95 phone driving loops on a 10-mile stretch of I-880 near Union City, California, for 8 hours. Data were collected using virtual trip lines, which are geographical markers stored in the handset that probabilistically trigger position and speed updates when the handset crosses them. The proposed prototype system provided sufficient data for traffic monitoring purposes while managing the privacy of participants. The data obtained in the experiment were processed in real-time and successfully broadcast on the internet, demonstrating the feasibility of the proposed system for real-time traffic monitoring. Results suggest that a 2-3% penetration of cell phones in the driver population is enough to provide accurate measurements of the velocity of the traffic flow.

Evaluation of Traffic Data Obtained via GPS-Enabled Mobile Phones: the Mobile Century Field Experiment

The growing need of the driving public for accurate traffic information has spurred the deployment of large scale dedicated monitoring infrastructure systems, which mainly consist in the use of inductive loop detectors and video cameras On-board electronic devices have been proposed as an alternative traffic sensing infrastructure, as they usually provide a cost-effective way to collect traffic data, leveraging existing communication infrastructure such as the cellular phone network A traffic monitoring system based on GPS-enabled smartphones exploits the extensive coverage provided by the cellular network, the high accuracy in position and velocity measurements provided by GPS devices, and the existing infrastructure of the communication network This article presents a field experiment nicknamed Mobile Century, which was conceived as a proof of concept of such a system Mobile Century included 100 vehicles carrying a GPS-enabled Nokia N95 phone driving loops on a 10-mile stretch of I-880 near Union City, California, for 8 hours Data were collected using virtual trip lines, which are geographical markers stored in the handset that probabilistically trigger position and speed updates when the handset crosses them The proposed prototype system provided sufficient data for traffic monitoring purposes while managing the privacy of participants The data obtained in the experiment were processed in real-time and successfully broadcast on the internet, demonstrating the feasibility of the proposed system for real-time traffic monitoring Results suggest that a 2-3% penetration of cell phones in the driver population is enough to provide accurate measurements of the velocity of the traffic flow

/pdf/evaluation-of-traffic-data-obtained-via-gps-enabled-mobile-3m3gqm5qzi.pdf

Evaluation of traffic data obtained via GPS-enabled mobile phones: The Mobile Century field experiment

The paper presents a statistical model for urban road network travel time estimation using vehicle trajectories obtained from low frequency GPS probes as observations, where the vehicles typically cover multiple network links between reports. The network model separates trip travel times into link travel times and intersection delays and allows correlation between travel times on different network links based on a spatial moving average (SMA) structure. The observation model presents a way to estimate the parameters of the network model, including the correlation structure, through low frequency sampling of vehicle traces. Link-specific effects are combined with link attributes (speed limit, functional class, etc.) and trip conditions (day of week, season, weather, etc.) as explanatory variables. The approach captures the underlying factors behind spatial and temporal variations in speeds, which is useful for traffic management, planning and forecasting. The model is estimated using maximum likelihood. The model is applied in a case study for the network of Stockholm, Sweden. Link attributes and trip conditions (including recent snowfall) have significant effects on travel times and there is significant positive correlation between segments. The case study highlights the potential of using sparse probe vehicle data for monitoring the performance of the urban transport system.

/pdf/travel-time-estimation-for-urban-road-networks-using-low-1o5qlb4jwd.pdf

Travel time estimation for urban road networks using low frequency probe vehicle data

Traffic state estimation on highway: A comprehensive survey

Traffic state estimation is a challenging problem for the transportation community due to the limited deployment of sensing infrastructure. However, recent trends in the mobile phone industry suggest that GPS equipped devices will become standard in the next few years. Leveraging these GPS equipped devices as traffic sensors will fundamentally change the type and the quality of traffic data collected on large scales in the near future. New traffic models and data assimilation algorithms must be developed to efficiently transform this data into usable traffic information. In this work, we introduce a new partial differential equation (PDE) based on the Lighthill-Whitham-Richards PDE, which serves as a flow model for velocity. We formulate a Godunov discretization scheme to cast the PDE into a Velocity Cell Transmission Model (CTM-v), which is a nonlinear dynamical system with a time varying observation matrix. The Ensemble Kalman Filtering (EnKF) technique is applied to the CTM- v to estimate the velocity field on the highway using data obtained from GPS devices, and the method is illustrated in microsimulation on a fully calibrated model of I880 in California. Experimental validation is performed through the unprecedented 100-vehicle Mobile Century experiment, which used a novel privacy-preserving traffic monitoring system to collect GPS cell phone data specifically for this research.

/pdf/an-ensemble-kalman-filtering-approach-to-highway-traffic-3j7t0eu48k.pdf

An ensemble Kalman filtering approach to highway traffic estimation using GPS enabled mobile devices

An inverse modeling algorithm is developed to reconstruct the state of traffic (velocity field) on highways from GPS measurements gathered from mobile phones traveling on-board vehicles. The algorithm is based on ensemble Kalman filtering (EnKF), to overcome the nonlinearity and non-differentiability of a distributed highway traffic model for velocity. The algorithm is implemented in an architecture which includes GPS enabled phones and a privacy aware data collection infrastructure based on the novel concept of Virtual Trip Lines (a technology developed by Nokia). The data collection infrastructure is connected to a traffic estimation server running the EnKF algorithm online, and the estimation results are broadcast in real time back to mobile phones and to the internet. Results from the algorithm are presented using data collected during the February 8, 2008 Mobile Century experiment, in which a shock wave from a five-car accident is captured. A prototype estimation algorithm and system were run during the experiment, and highlight that measurements from as few as 2% to 5% of the commuting public are sufficient to accurately reconstruct the highway traffic state.

Lagrangian sensing: traffic estimation with mobile devices

This article addresses the trade-offs between (i) velocity data collected from GPS smartphones in probe vehicles, and (ii), velocity data obtained from inductive loop detectors, for the purpose of computing travel times on a stretch of roadway. It is a case study which uses experimental data collected on one day in the San Francisco Bay Area, obtained as part of a 2008 field experiment known as Mobile Century. Estimates of the traffic velocity field are constructed using a velocity model equivalent to the Cell Transmission Model, and a traffic state estimation algorithm known as ensemble Kalman filtering. More than 900 combinations of inductive loop detector data and GPS data of various data volumes are considered. Instantaneous and a posteriori dynamic travel times are computed by integrating the estimated velocity field, and are compared to travel times recorded using license plate re-identification. Key findings include (i) dynamic travel time estimates can be computed with less than 10% error using only probe data, only inductive loop data, or a mixture of inductive loop and probe data (ii) by combining loop detector data with probe vehicle data, better estimates for travel times are obtained especially when loop data or probe data is sparse.

Trade-offs Between Inductive Loops and GPS Probe Vehicles for Travel Time Estimation: Mobile Century Case Study

We present a state estimation method for two-dimensional shallow water equations in rivers using Lagrangian drifter positions as measurements The aim of this method is to compensate for the lack of knowledge of upstream and downstream boundary conditions in rivers that causes inaccuracy in the velocity field estimation by releasing drifters equipped with GPS receivers The drifters report their positions and thus provide additional information of the state of the river This information is incorporated into shallow water equations by using Ensemble Kalman Filtering (EnKF) The proposed method is based on the discretization of the governing nonlinear equations using the finite element method in unstructured meshes We incorporate the drifter positions into the unknown state, which directly exploits the Langrangian nature of the measurements The performance of the method is assessed with twin experiments

/pdf/ensemble-kalman-filter-based-state-estimation-in-2d-shallow-5gpbwevqf7.pdf

Ensemble Kalman Filter based state estimation in 2D shallow water equations using Lagrangian sensing and state augmentation

This article presents the comparison of two algorithms for data assimilation of two
 dimensional shallow water flows. The first algorithm is based on a
 linearization of the model equations and a quadratic programming (QP)
 formulation of the problem. The second algorithm uses Ensemble
 Kalman Filtering (EnKF) applied to the non-linear two dimensional
 shallow water equations. The two methods are implemented
 on a scenario in which boundary conditions and Lagrangian
 measurements are available. The performance of the methods is
 evaluated using twin experiments with experimentally measured
 bathymetry data and boundary conditions from a river located in the
 Sacramento Delta. The sensitivity of the algorithms to the number of
 drifters, low or high discharge and time sampling frequency is
 studied.

Olli-Pekka Tossavainen

Papers

An ensemble Kalman filtering approach to highway traffic estimation using GPS enabled mobile devices

Lagrangian sensing: traffic estimation with mobile devices

Trade-offs Between Inductive Loops and GPS Probe Vehicles for Travel Time Estimation: Mobile Century Case Study

Ensemble Kalman Filter based state estimation in 2D shallow water equations using Lagrangian sensing and state augmentation

Comparison of two data assimilation algorithms for shallow water flows