Operational solar flare prediction model using Deep Flare Net.
TL;DR: In this article, the authors developed an operational solar flare prediction model using deep neural networks, named Deep Flare Net (DeFN), which can issue probabilistic forecasts of solar flares in two categories, such as >=M-class and =C-class.
Abstract: We developed an operational solar flare prediction model using deep neural networks, named Deep Flare Net (DeFN). DeFN can issue probabilistic forecasts of solar flares in two categories, such as >=M-class and =C-class and =M-class flares and TSS = 0.63 for >=C-class flares. For comparison, we evaluated the operationally forecast results from January 2019 to June 2020. We found that operational DeFN forecasts achieved TSS = 0.70 (0.84) for >=C-class flares with the probability threshold of 50 (40)%, although there were very few M-class flares during this period and we should continue monitoring the results for a longer time. Here, we adopted a chronological split to divide the database into two for training and testing. The chronological split appears suitable for evaluating operational models. Furthermore, we proposed the use of time-series cross-validation. The procedure achieved TSS = 0.70 for >=M-class flares and 0.59 for >=C-class flares using the datasets obtained from 2010 to 2017.
Citations
More filters
TL;DR: In this article , the authors proposed a general paradigm to generate these sets in such a way that they are independent from each other and internally well-balanced in terms of AR flaring effectiveness.
Abstract: Solar flare forecasting can be realized by means of the analysis of magnetic data through artificial intelligence techniques. The aim is to predict whether a magnetic active region (AR) will originate solar flares above a certain class within a certain amount of time. A crucial issue is concerned with the way the adopted machine learning method is implemented, since forecasting results strongly depend on the criterion with which training, validation, and test sets are populated. In this paper we propose a general paradigm to generate these sets in such a way that they are independent from each other and internally well-balanced in terms of AR flaring effectiveness. This set generation process provides a ground for comparison for the performance assessment of machine learning algorithms. Finally, we use this implementation paradigm in the case of a deep neural network, which takes as input videos of magnetograms recorded by the Helioseismic and Magnetic Imager on-board the Solar Dynamics Observatory (SDO/HMI). To our knowledge, this is the first time that the solar flare forecasting problem is addressed by means of a deep neural network for video classification, which does not require any a priori extraction of features from the HMI magnetograms.
4 citations
TL;DR: In this paper , a specific type of recurrent neural network (encoder-decoder) is used to construct global electron density based on the multi-year data from Van Allen Probes.
Abstract: The total electron density is a fundamental quantity in the Earth's magnetosphere and plays an important role in a number of physical processes, but its dynamic global evolution is not fully quantified yet. We present an implementation of a specific type of recurrent neural network (encoder-decoder), which is distinct from previous models, to construct global electron density based on the multiyear data from Van Allen Probes. The history of geomagnetic indices is first encoded into a hidden state H, then together with auxiliary information (satellite location), they are decoded into the quantity of interest (total electron density in this study). In this process the input of historical geomagnetic indices is detangled from the satellite location and is processed chronologically by the encoder. As a result, time evolution of geomagnetic indices is explicitly embedded in the structure and the encoded hidden state H can be viewed as the representation of the inner magnetospheric state. The magnetospheric state is then decoded to predict global electron density evolution. Our results show that the model can capture the dynamical evolution of total electron density with the formation and evolution of stable and evident plume configurations that roughly agree with global observations. Our findings demonstrate the importance of applying recurrent neural networks to specify the inner magnetospheric state in a novel way, which will potentially improve our fundamental understanding of wave and particle dynamics in the Earth's magnetosphere.
2 citations
TL;DR: This study shows that video-based deep learning can be used for operational purposes when the training and validation sets used for the network optimization are generated while accounting for the periodicity of the solar cycle.
Abstract: Operational flare forecasting aims at providing predictions that can be used to make decisions, typically on a daily scale, about the space weather impacts of flare occurrence. This study shows that video-based deep learning can be used for operational purposes when the training and validation sets used for network optimization are generated while accounting for the periodicity of the solar cycle. Specifically, this article describes an algorithm that can be applied to build up sets of active regions that are balanced according to the flare class rates associated to a specific cycle phase. These sets are used to train and validate a long-term recurrent convolutional network made of a combination of a convolutional neural network and a long short-term memory network. The reliability of this approach is assessed in the case of two prediction windows containing the solar storms of March 2015, June 2015, and September 2017.
2 citations
TL;DR: A set of new heuristic approaches to train and deploy an operational solar flare prediction system for ≥M1.0-class flares with two prediction modes: full-disk and active region-based using deep learning models.
Abstract: Solar flare prediction is a central problem in space weather forecasting and has captivated the attention of a wide spectrum of researchers due to recent advances in both remote sensing as well as machine learning and deep learning approaches. The experimental findings based on both machine and deep learning models reveal significant performance improvements for task specific datasets. Along with building models, the practice of deploying such models to production environments under operational settings is a more complex and often time-consuming process which is often not addressed directly in research settings. We present a set of new heuristic approaches to train and deploy an operational solar flare prediction system for ≥M1.0-class flares with two prediction modes: full-disk and active region-based. In full-disk mode, predictions are performed on full-disk line-of-sight magnetograms using deep learning models whereas in active region-based models, predictions are issued for each active region individually using multivariate time series data instances. The outputs from individual active region forecasts and full-disk predictors are combined to a final full-disk prediction result with a meta-model. We utilized an equal weighted average ensemble of two base learners’ flare probabilities as our baseline meta learner and improved the capabilities of our two base learners by training a logistic regression model. The major findings of this study are: 1) We successfully coupled two heterogeneous flare prediction models trained with different datasets and model architecture to predict a full-disk flare probability for next 24 h, 2) Our proposed ensembling model, i.e., logistic regression, improves on the predictive performance of two base learners and the baseline meta learner measured in terms of two widely used metrics True Skill Statistic (TSS) and Heidke Skill Score (HSS), and 3) Our result analysis suggests that the logistic regression-based ensemble (Meta-FP) improves on the full-disk model (base learner) by ∼9% in terms TSS and ∼10% in terms of HSS. Similarly, it improves on the AR-based model (base learner) by ∼17% and ∼20% in terms of TSS and HSS respectively. Finally, when compared to the baseline meta model, it improves on TSS by ∼10% and HSS by ∼15%.
2 citations
TL;DR: Based on the distribution properties of strong-flare occurrences related to SHARP parameters, this article established a new selective up-sampling method and applied it to the mixed-up region to pick up the flare-related samples and add small random values to them and finally create a large number of flarerelated samples that are very close to the ground truth.
Abstract: The Spaceweather HMI Active Region Patch (SHARP) parameters have been widely used to develop flare prediction models. The relatively small number of strong-flare events leads to an unbalanced dataset that prediction models can be sensitive to the unbalanced data and might lead to bias and limited performance. In this study, we adopted the logistic regression algorithm to develop a flare prediction model for the next 48 h based on the SHARP parameters. The model was trained with five different inputs. The first input was the original unbalanced dataset; the second and third inputs were obtained by using two widely used sampling methods from the original dataset, while the fourth input was the original dataset but accompanied by a weighted classifier. Based on the distribution properties of strong-flare occurrences related to SHARP parameters, we established a new selective up-sampling method and applied it to the mixed-up region (referred to as the confusing distribution areas consisting of both the strong-flare events and non-strong-flare events) to pick up the flare-related samples and add small random values to them and finally create a large number of flare-related samples that are very close to the ground truth. Thus, we obtained the fifth balanced dataset aiming to 1) promote the forecast capability in the mixed-up region and 2) increase the robustness of the model. We compared the model performance and found that the selective up-sampling method has potential to improve the model performance in strong-flare prediction with its F1 score reaching 0.5501 ± 0.1200, which is approximately 22% − 33% higher than other imbalance mitigation schemes.
1 citations
References
More filters
Proceedings Article•
01 Jan 2015TL;DR: This work introduces Adam, an algorithm for first-order gradient-based optimization of stochastic objective functions, based on adaptive estimates of lower-order moments, and provides a regret bound on the convergence rate that is comparable to the best known results under the online convex optimization framework.
Abstract: We introduce Adam, an algorithm for first-order gradient-based optimization of stochastic objective functions, based on adaptive estimates of lower-order moments. The method is straightforward to implement, is computationally efficient, has little memory requirements, is invariant to diagonal rescaling of the gradients, and is well suited for problems that are large in terms of data and/or parameters. The method is also appropriate for non-stationary objectives and problems with very noisy and/or sparse gradients. The hyper-parameters have intuitive interpretations and typically require little tuning. Some connections to related algorithms, on which Adam was inspired, are discussed. We also analyze the theoretical convergence properties of the algorithm and provide a regret bound on the convergence rate that is comparable to the best known results under the online convex optimization framework. Empirical results demonstrate that Adam works well in practice and compares favorably to other stochastic optimization methods. Finally, we discuss AdaMax, a variant of Adam based on the infinity norm.
111,197 citations
Book•
17 Aug 2006TL;DR: Probability Distributions, linear models for Regression, Linear Models for Classification, Neural Networks, Graphical Models, Mixture Models and EM, Sampling Methods, Continuous Latent Variables, Sequential Data are studied.
Abstract: Probability Distributions.- Linear Models for Regression.- Linear Models for Classification.- Neural Networks.- Kernel Methods.- Sparse Kernel Machines.- Graphical Models.- Mixture Models and EM.- Approximate Inference.- Sampling Methods.- Continuous Latent Variables.- Sequential Data.- Combining Models.
22,840 citations
TL;DR: The Atmospheric Imaging Assembly (AIA) as discussed by the authors provides multiple simultaneous high-resolution full-disk images of the corona and transition region up to 0.5 R ⊙ above the solar limb with 1.5-arcsec spatial resolution and 12-second temporal resolution.
Abstract: The Atmospheric Imaging Assembly (AIA) provides multiple simultaneous high-resolution full-disk images of the corona and transition region up to 0.5 R ⊙ above the solar limb with 1.5-arcsec spatial resolution and 12-second temporal resolution. The AIA consists of four telescopes that employ normal-incidence, multilayer-coated optics to provide narrow-band imaging of seven extreme ultraviolet (EUV) band passes centered on specific lines: Fe xviii (94 A), Fe viii, xxi (131 A), Fe ix (171 A), Fe xii, xxiv (193 A), Fe xiv (211 A), He ii (304 A), and Fe xvi (335 A). One telescope observes C iv (near 1600 A) and the nearby continuum (1700 A) and has a filter that observes in the visible to enable coalignment with images from other telescopes. The temperature diagnostics of the EUV emissions cover the range from 6×104 K to 2×107 K. The AIA was launched as a part of NASA’s Solar Dynamics Observatory (SDO) mission on 11 February 2010. AIA will advance our understanding of the mechanisms of solar variability and of how the Sun’s energy is stored and released into the heliosphere and geospace.
4,321 citations
TL;DR: The Solar Dynamics Observatory (SDO) was launched on 11 February 2010 at 15:23 UT from Kennedy Space Center aboard an Atlas V 401 (AV-021) launch vehicle as mentioned in this paper.
Abstract: The Solar Dynamics Observatory (SDO) was launched on 11 February 2010 at 15:23 UT from Kennedy Space Center aboard an Atlas V 401 (AV-021) launch vehicle. A series of apogee-motor firings lifted SDO from an initial geosynchronous transfer orbit into a circular geosynchronous orbit inclined by 28° about the longitude of the SDO-dedicated ground station in New Mexico. SDO began returning science data on 1 May 2010. SDO is the first space-weather mission in NASA’s Living With a Star (LWS) Program. SDO’s main goal is to understand, driving toward a predictive capability, those solar variations that influence life on Earth and humanity’s technological systems. The SDO science investigations will determine how the Sun’s magnetic field is generated and structured, how this stored magnetic energy is released into the heliosphere and geospace as the solar wind, energetic particles, and variations in the solar irradiance. Insights gained from SDO investigations will also lead to an increased understanding of the role that solar variability plays in changes in Earth’s atmospheric chemistry and climate. The SDO mission includes three scientific investigations (the Atmospheric Imaging Assembly (AIA), Extreme Ultraviolet Variability Experiment (EVE), and Helioseismic and Magnetic Imager (HMI)), a spacecraft bus, and a dedicated ground station to handle the telemetry. The Goddard Space Flight Center built and will operate the spacecraft during its planned five-year mission life; this includes: commanding the spacecraft, receiving the science data, and forwarding that data to the science teams. The science investigations teams at Stanford University, Lockheed Martin Solar Astrophysics Laboratory (LMSAL), and University of Colorado Laboratory for Atmospheric and Space Physics (LASP) will process, analyze, distribute, and archive the science data. We will describe the building of SDO and the science that it will provide to NASA.
3,043 citations
TL;DR: The Helioseismic and Magnetic Imager (HMI) instrument and investigation as a part of the NASA Solar Dynamics Observatory (SDO) is designed to study convection-zone dynamics and the solar dynamo, the origin and evolution of sunspots, active regions, and complexes of activity, the sources and drivers of solar magnetic activity and disturbances as mentioned in this paper.
Abstract: The Helioseismic and Magnetic Imager (HMI) instrument and investigation as a part of the NASA Solar Dynamics Observatory (SDO) is designed to study convection-zone dynamics and the solar dynamo, the origin and evolution of sunspots, active regions, and complexes of activity, the sources and drivers of solar magnetic activity and disturbances, links between the internal processes and dynamics of the corona and heliosphere, and precursors of solar disturbances for space-weather forecasts. A brief overview of the instrument, investigation objectives, and standard data products is presented.
2,242 citations