Showing papers on "Nowcasting published in 2023"

Reconciled Estimates of Monthly GDP in the United States

Abstract: Abstract In the United States, income and expenditure-side estimates of gross domestic product (GDP) (GDP and GDP ) measure “true” GDP with error and are available at a quarterly frequency. Methods exist for using these proxies to produce reconciled quarterly estimates of true GDP. In this paper, we extend these methods to provide reconciled historical true GDP estimates at a monthly frequency. We do this using a Bayesian mixed frequency vector autoregression (MF-VAR) involving GDP , GDP , unobserved true GDP, and monthly indicators of short-term economic activity. Our MF-VAR imposes restrictions that reflect a measurement-error perspective (i.e., the two GDP proxies are assumed to equal true GDP plus measurement error). Without further restrictions, our model is unidentified. We consider a range of restrictions that allow for point and set identification of true GDP and show that they lead to informative monthly GDP estimates. We illustrate how these new monthly data contribute to our historical understanding of business cycles and we provide a real-time application nowcasting monthly GDP over the pandemic recession.

Real-time forecasting of time series in financial markets using sequentially trained dual-LSTMs

Abstract: Financial markets are highly complex and volatile; thus, accurate forecasting of such markets is vital to make early alerts about crashes and subsequent recoveries. People have been using learning tools from diverse fields such as financial mathematics and machine learning to make trustworthy forecasting on such markets. However, the accuracy of such techniques had not been adequate until artificial neural network frameworks such as long short-term memory (LSTM) were utilized. Moreover, making accurate real-time forecasting, also known as nowcasting, of financial time series is highly subjective to the LSTM’s architecture in use and the procedure of training it. Herein, we forecast financial markets in real-time by training a dual version of LSTM which forecasts only one time step at each iteration so that the forecast for this iteration will be in the input for the next iteration. Semi-convergence is a prominent issue in a recurrent LSTM setup as the error could propagate through iterations; however, the duality of this LSTM aids in dwindling this issue. Especially, we employ one LSTM to find the best number of epochs associated with the least loss and train the second LSTM only through that many epochs to make forecasting. We treat the current forecast as a part of the training set for the next forecast and train the same LSTM. While classic ways of training cause more error when the forecast is made further away through the test period, our approach offers superior accuracy as the training increases when it proceeds through the testing period. The forecasting accuracy of our approach is validated using three time series from each of the three diverse financial markets: stock, cryptocurrency, and commodity. The results are compared with those of a single LSTM, an extended Kalman filter, and an autoregressive integrated moving average model.

Nowcasting Daily Population Displacement in Ukraine through Social Media Advertising Data

Abstract: In times of crisis, real-time data mapping population displacements are invaluable for targeted humanitarian response. The Russian invasion of Ukraine on February 24, 2022, forcibly displaced millions of people from their homes including nearly 6 million refugees flowing across the border in just a few weeks, but information was scarce regarding displaced and vulnerable populations who remained inside Ukraine. We leveraged social media data from Facebook's advertising platform in combination with preconflict population data to build a real-time monitoring system to estimate subnational population sizes every day disaggregated by age and sex. Using this approach, we estimated that 5.3 million people had been internally displaced away from their baseline administrative region in the first three weeks after the start of the conflict. Results revealed four distinct displacement patterns: large-scale evacuations, refugee staging areas, internal areas of refuge, and irregular dynamics. While the use of social media provided one of the only quantitative estimates of internal displacement in the conflict setting in virtual real time, we conclude by acknowledging risks and challenges of these new data streams for the future.

3D-UNet-LSTM: A Deep Learning-Based Radar Echo Extrapolation Model for Convective Nowcasting

Abstract: Radar echo extrapolation is a commonly used approach for convective nowcasting. The evolution of convective systems over a very short term can be foreseen according to the extrapolated reflectivity images. Recently, deep neural networks have been widely applied to radar echo extrapolation and have achieved better forecasting performance than traditional approaches. However, it is difficult for existing methods to combine predictive flexibility with the ability to capture temporal dependencies at the same time. To leverage the advantages of the previous networks while avoiding the mentioned limitations, a 3D-UNet-LSTM model, which has an extractor-forecaster architecture, is proposed in this paper. The extractor adopts 3D-UNet to extract comprehensive spatiotemporal features from the input radar images. In the forecaster, a newly designed Seq2Seq network exploits the extracted features and uses different convolutional long short-term memory (ConvLSTM) layers to iteratively generate hidden states for different future timestamps. Finally, the hidden states are transformed into predicted radar images through a convolutional layer. We conduct 0–1 h convective nowcasting experiments on the public MeteoNet dataset. Quantitative evaluations demonstrate the effectiveness of the 3D-UNet extractor, the newly designed forecaster, and their combination. In addition, case studies qualitatively demonstrate that the proposed model has a better spatiotemporal modeling ability for the complex nonlinear processes of convective echoes.

Sensitivity to Localization Radii for an Ensemble Filter Numerical Weather Prediction System with 30-Second Update

Abstract: A sensitivity analysis for horizontal localization scale is performed for a numerical weather prediction (NWP) system that uses a 30-second update to refresh a 500-m mesh with observations from a new-generation multi-parameter phased array weather radar (MP-PAWR). Testing is performed using three case studies of convective weather events that occurred during August/September 2019, with the aim to determine the most suitable scale for short-range forecasting of precipitating convective systems and better understand model behavior to a rapid update cycle. Results showed that while the model could provide useful skill at lead times up to 30-minutes, forecasts would consistently over-estimate rainfall and were unable to outperform nowcasts performed with a simple advection model. Using a larger localization scale e.g., 4-km, generated stronger convective and dynamical instability in the analyzes that made conditions more favorable for spurious and intense convection to develop in forecasts. It was demonstrated that lowering the localization scale reduced the size of analysis increments during early cycling, limiting the buildup of these conditions. Improved representation of the localized convection in the initial conditions was suggested as an important step to mitigating this issue in the model.

Improving Solar Radiation Nowcasts by Blending Data-Driven, Satellite-Images-Based and All-Sky-Imagers-Based Models Using Machine Learning Techniques

Abstract: Accurate solar radiation nowcasting models are critical for the integration of the increasing solar energy in power systems. This work explored the benefits obtained by the blending of four all-sky-imagers (ASI)-based models, two satellite-images-based models and a data-driven model. Two blending approaches (general and horizon) and two blending models (linear and random forest (RF)) were evaluated. The relative contribution of the different forecasting models in the blended-models-derived benefits was also explored. The study was conducted in Southern Spain; blending models provide one-minute resolution 90 min-ahead GHI and DNI forecasts. The results show that the general approach and the RF blending model present higher performance and provide enhanced forecasts. The improvement in rRMSE values obtained by model blending was up to 30% for GHI (40% for DNI), depending on the forecasting horizon. The greatest improvement was found at lead times between 15 and 30 min, and was negligible beyond 50 min. The results also show that blending models using only the data-driven model and the two satellite-images-based models (one using high resolution images and the other using low resolution images) perform similarly to blending models that used the ASI-based forecasts. Therefore, it was concluded that suitable model blending might prevent the use of expensive (and highly demanding, in terms of maintenance) ASI-based systems for point nowcasting.

The Impact of COVID-19 on Living Standards: Addressing the Challenges of Nowcasting Unprecedented Macroeconomic Shocks with Scant Data and Uncharted Economic Behavior

Abstract: We present a methodological approach with relatively low information requirements to quantify the impact of large, unprecedented macroeconomic shocks like the COVID19 pandemic on living standards across the income distribution. The approach can be produced quickly and, contrary to other "fastdelivery" exercises, does not assume that income losses are proportional across the income distribution, a feature that is critical to understanding the impact on poverty and inequality. Our method is sufficiently flexible to refine the projected effects of the shock as more information becomes available. We illustrate with data from the four largest countries in Latin America: Argentina, Brazil, Colombia, and Mexico, and discuss the estimated effect of COVID19 on inequality and poverty. We also present the guidelines for adapting our framework to different countries and economic shocks. JEL classification: C63, D31, E27, I32, I38 DOI: https:// doi. org/ 10. 34196/ ijm. 00273

Nowcasting COVID-19 Statistics Reported with Delay: A Case-Study of Sweden and the UK

Abstract: The COVID-19 pandemic has demonstrated the importance of unbiased, real-time statistics of trends in disease events in order to achieve an effective response. Because of reporting delays, real-time statistics frequently underestimate the total number of infections, hospitalizations and deaths. When studied by event date, such delays also risk creating an illusion of a downward trend. Here, we describe a statistical methodology for predicting true daily quantities and their uncertainty, estimated using historical reporting delays. The methodology takes into account the observed distribution pattern of the lag. It is derived from the “removal method”—a well-established estimation framework in the field of ecology.

Earthquake Nowcasting: Retrospective Testing in Greece

Abstract: Earthquake nowcasting (EN) is a modern method of estimating seismic risk by evaluating the progress of the earthquake (EQ) cycle in fault systems. EN evaluation is based on a new concept of time, termed ’natural time’. EN employs natural time, and uniquely estimates seismic risk by means of the earthquake potential score (EPS), which has been found to have useful applications both regionally and globally. Amongst these applications, here we focused on Greece since 2019, for the estimation of the EPS for the largest-magnitude events, MW(USGS) ≥ 6, that occurred during our study period: for example, the MW= 6.0 WNW-of-Kissamos EQ on 27 November 2019, the MW= 6.5 off-shore Southern Crete EQ on 2 May 2020, the MW= 7.0 Samos EQ on 30 October 2020, the MW= 6.3 Tyrnavos EQ on 3 March 2021, the MW= 6.0 Arkalohorion Crete EQ on 27 September 2021, and the MW= 6.4 Sitia Crete EQ on 12 October 2021. The results are promising, and reveal that the EPS provides useful information on impending seismicity.

Structural Forecasting for Short-term Tropical Cyclone Intensity Guidance

Abstract: Because geostationary satellite (Geo) imagery provides a high temporal resolution window into tropical cyclone (TC) behavior, we investigate the viability of its application to short-term probabilistic forecasts of TC convective structure to subsequently predict TC intensity. Here, we present a prototype model which is trained solely on two inputs: Geo infrared imagery leading up to the synoptic time of interest and intensity estimates up to 6 hours prior to that time. To estimate future TC structure, we compute cloud-top temperature radial profiles from infrared imagery and then simulate the evolution of an ensemble of those profiles over the subsequent 12 hours by applying a Deep Autoregressive Generative Model (PixelSNAIL). To forecast TC intensities at hours 6 and 12, we input operational intensity estimates up to the current time (0 h) and simulated future radial profiles up to +12 h into a ``nowcasting'' convolutional neural network. We limit our inputs to demonstrate the viability of our approach and to enable quantification of value added by the observed and simulated future radial profiles beyond operational intensity estimates alone. Our prototype model achieves a marginally higher error than the National Hurricane Center's official forecasts despite excluding environmental factors, such as vertical wind shear and sea surface temperature. We also demonstrate that it is possible to reasonably predict short-term evolution of TC convective structure via radial profiles from Geo infrared imagery, resulting in interpretable structural forecasts that may be valuable for TC operational guidance.

Comparison of three radar-based precipitation nowcasts for the extreme July 2021 flooding event in Germany

Abstract: Quantitative precipitation nowcasts (QPN) can improve the accuracy of flood forecasts especially for lead times up to 12 hours, but their evaluation depends on a variety of factors, namely the choice of the hydrological model and the benchmark. We tested three precipitation nowcasting techniques based on radar observations for the disastrous mid-July 2021 event in seven German catchments (140-1670 km2). Two deterministic (advection-based and S-PROG) and one probabilistic (STEPS) QPN with maximum lead time of 3 h were used as input to two hydrological models: a physically-based, 3D-distributed model (ParFlowCLM) and a conceptual, lumped model (GR4H). We quantified the hydrological added value of QPN compared to hydrological persistence and zero-precipitation nowcasts as benchmarks. For the 14 July 2021 event, we obtained the following key results: (1) According to the quality of the forecasted hydrographs, exploiting QPN improved the lead times by up to 4 h (8 h) compared to adopting zero-precipitation nowcasts (hydrological persistence) as a benchmark. Using a skill-based approach, obtained improvements were up to 7-12 h depending on the benchmark. (2) The three QPN techniques obtained similar performances regardless of the applied hydrological model. (3) Using zero-precipitation nowcasts instead of hydrological persistence as benchmark reduced the added value of QPN. These results highlight the need for combining a skill-based approach with an analysis of the quality of forecasted hydrographs to rigorously estimate the added value of QPN.

Fog in Sofia 2010–2019: Objective Circulation Classification and Fog Indices

Abstract: Low visibility caused by fog events can lead to disruption of every type of public transportation, and even loss of life. The focus of this study is the synoptic conditions associated with fog formation. The data used in this study was collected over the course of ten years (2010–2019) in Sofia, Bulgaria. The forecast skills of the Fog Stability Index (FSI) and the local Sofia Stability Index (SSI), as well as the relation between the Integrated Water Vapor (IWV) and fog from the Global Navigation Satellite System (GNSS), were tested. Both fog indices are used for fog nowcasting as their lead times are short and unclear. The Jenkinson–Collison Type method was used for extracting the predominant synoptic-scale pressure systems which provide suitable weather conditions for fog formation. Surface observations from two synoptic stations were used to calculate and evaluate the performance of the two fog indices and of the ground-based GNSS receiver for the IWV. The forecast skills provided by Probability of Detection (POD) and False Alarm Ratio (FAR), for both fog and no-fog periods, were obtained by discriminant analysis. Additionally, several weather parameters, such as surface wind speed, relative humidity and IWV, were added in order to improve the results of the local index (SSI). This led to a 77.9% hit rate. The cyclonic system influence and zonal flows from the west and the southwest are both responsible for a number of fog cases that are comparable to those associated with the anticyclonic system. The IWV was not found to improve the forecast skill of the fog indices. However, it was found that its values had a larger spread during no-fog periods in comparison to fog periods.

Nowcasting and Forecasting the 2022 U.S. Mpox Outbreak: Support for Public Health Decision Making and Lessons Learned

Abstract: Mpox is a zoonotic disease endemic in Central and West Africa. In May 2022, an outbreak of mpox characterized by human-to-human transmission was detected in multiple non-endemic countries. We performed nowcasting and forecasting for the 2022 mpox outbreak in the United States using the R package EpiNow2. We generated nowcasts/forecasts at the national level, by Census region, and for jurisdictions reporting the greatest number of mpox cases. Modeling results were shared for situational awareness within the Centers for Disease Control and Prevention (CDC) Mpox Response and publicly on the CDC website. We retrospectively evaluated forecast predictions at four key phases during the outbreak using three metrics, the weighted interval score, mean absolute error, and prediction interval coverage. We compared the performance of EpiNow2 with a naive Bayesian generalized linear model (GLM). The EpiNow2 model had less probabilistic error than the GLM during every outbreak phase except for the early phase. We share our experiences with an existing tool for nowcasting/forecasting and highlight areas of improvement for the development of future tools. We also reflect on lessons learned regarding data quality issues and adapting modeling results for different audiences.

Word embeddings for topic modeling: An application to the estimation of the economic policy uncertainty index

Abstract: The quantification of economic uncertainty is key to the prediction of macroeconomic variables, such as gross domestic product (GDP), and is particularly crucial in regard to real-time or short-time prediction methodologies, such as nowcasting, where a large amount of time series data is required. Most of the data comes from official agency statistics and non-public institutions, but these sources are susceptible to lack of information due to major disruptive events, such as the COVID-19 pandemic. Because of this, it is very common nowadays to use non-traditional data from different sources. The economic policy uncertainty (EPU) index is the indicator most frequently used to quantify uncertainty and is based on topic modeling of newspapers. In this paper, we propose a methodology to estimate the EPU index that incorporates a fast and efficient method for topic modeling of digital news based on semantic clustering with word embeddings, allowing us to update the index in real time, which is something that other studies have failed to manage. We show that our proposal enables us to update the index and significantly reduce the time required for new document assignation into topics.

IMPROVER: the new probabilistic post processing system at the UK Met Office

Abstract: The Met Office in the UK has developed a completely new probabilistic post-processing system called IMPROVER to operate on outputs from its operational Numerical Weather Prediction (NWP) forecasts and precipitation nowcasts. The aim is to improve weather forecast information to the public and other stakeholders whilst better exploiting the current and future generations of underpinning kilometer-scale NWP ensembles. We wish to provide seamless forecasts from nowcasting to medium range, provide consistency between gridded and site-specific forecasts and be able to verify every stage of the processing. The software is written in a modern modular framework that is easy to maintain, develop and share. IMPROVER allows forecast information to be provided with greater spatial and temporal detail and a faster update frequency than previous post-processing. Independent probabilistic processing chains are constructed for each meteorological variable consisting of a series of processing stages that operate on pre-defined grids and blend outputs from several NWP inputs to give a frequently updated, probabilistic forecast solution. Probabilistic information is produced as standard, with the option of extracting a most likely or yes/no outcome if required. Verification can be performed at all stages, although it is only currently switched on for the most significant stages when run in real time. IMPROVER has been producing real-time output since March 2021 and became operational in Spring 2022.

Nowcasting inflation with Lasso‐regularized vector autoregressions and mixed frequency data

Abstract: We evaluate the predictive performances of the least absolute shrinkage and selection operator (Lasso) as an alternative shrinkage method for high-dimensional vector autoregressions. The analysis extends the Lasso-based multiple equations regularization to a mixed/high-frequency data setting. Very short-term forecasting (nowcasting) is used to target the Euro area's inflation rate. We show that this approach can outperform more standard nowcasting tools in the literature, producing nowcasts that closely follow actual data movements. The proposed tool can overcome information and policy decision problems related to the substantial publishing delays of macroeconomic aggregates.