Multi-Channel Auto-Calibration for the Atmospheric Imaging Assembly using Machine Learning

TL;DR: In this article, an alternative calibration approach based on convolutional neural networks (CNNs) was proposed to calibrate Extreme UV (EUV) wavelength observations from space, which can comprehensively reproduce the sounding rocket experiments' outcomes within a reasonable degree of accuracy.

Abstract: Solar activity plays a quintessential role in influencing the interplanetary medium and space-weather around the Earth. Remote sensing instruments onboard heliophysics space missions provide a pool of information about the Sun's activity via the measurement of its magnetic field and the emission of light from the multi-layered, multi-thermal, and dynamic solar atmosphere. Extreme UV (EUV) wavelength observations from space help in understanding the subtleties of the outer layers of the Sun, namely the chromosphere and the corona. Unfortunately, such instruments, like the Atmospheric Imaging Assembly (AIA) onboard NASA's Solar Dynamics Observatory (SDO), suffer from time-dependent degradation, reducing their sensitivity. Current state-of-the-art calibration techniques rely on periodic sounding rockets, which can be infrequent and rather unfeasible for deep-space missions. We present an alternative calibration approach based on convolutional neural networks (CNNs). We use SDO-AIA data for our analysis. Our results show that CNN-based models could comprehensively reproduce the sounding rocket experiments' outcomes within a reasonable degree of accuracy, indicating that it performs equally well compared with the current techniques. Furthermore, a comparison with a standard "astronomer's technique" baseline model reveals that the CNN approach significantly outperforms this baseline. Our approach establishes the framework for a novel technique to calibrate EUV instruments and advance our understanding of the cross-channel relation between different EUV channels.

Summary

1. Introduction

• Solar activity plays a significant role in affecting the interplanetary medium and space weather around Earth and all the other planets of the Solar System (Schwenn 2006).
• One of the flagship missions of HSO is the Solar Dynamics Observatory (SDO, Pesnell et al. 2012).
• The top row shows the images observed during the early days of the mission, from 13 May 2010, and the bottom row shows the corresponding images observed more recently on 31 August 2019, scaled within the same intensity range.
• The dimming effect observed in the channels is due to the temporal degradation of EUV instruments in space that is also known to diminish the overall instrument sensitivity with time (e.g., BenMoussa et al. 2013).
• A comparison of the sounding-rocket observation with the satellite instrument observation provides an updated calibration, revealing long-term trends in the sensitivities of EVE and thus of AIA.

2. Data description and preprocessing

• The authors used the preprocessed SDO-AIA dataset from Galvez et al. (2019, hereafter referred to as SDOML).
• The data from the two SDO instruments are temporally aligned, with cadences of 6 min for AIA (instead of the original 12 s) and EVE and 12 min for HMI.
• The authors established that 256 × 256 is a sufficient resolution for the predictive task of interest (inference of a single coefficient), and the reduced size enabled quicker processing and more efficient use of the computational resources.
• These factors represent the approximate average AIA data counts in each channel and in the period from 2011 to 2018 (derived from Galvez et al. 2019).

3.1. Formulation of the problem

• It is known that some bright structures in the Sun are observed at different wavelengths.
• Based on this cross-channel structure, the authors established a hypothesis divided into two parts.
• Second, that this relation of the morphological features and the brightness of solar structures can be found in multiple AIA channels.
• The authors hypothesize that these two relations can be used to estimate the dimming factors, and that a deep-learning model can automatically learn these inter- and cross-channel patterns and exploit them to accurately predict the dimming factor of each channel.
• The spatial dependence of the degradation is assumed to be accounted for by regularly updated flat-field corrections applied to AIA images.

3.2. Convolutional neural network model

• Deep learning is a highly active subfield of ML that focuses on specific models called deep neural networks (DNNs).
• DNNs have produced the state-of-art results in many complex tasks, including object detection in images (He et al. 2016), speech recognition (Amodei et al. 2016) and synthesis (Oord et al. 2016), and translation between languages (Wu et al. 2016).
• This architecture is a test of the first hypothesis in Sect. 3.1. The second architecture is instead designed to exploit possible cross-channel relations while training, and it tests their second hypothesis: solar surface features that appear in the different channels will make a multichannel CNN architecture more effective than a single-channel CNN that only exploits interchannel structure correlations.
• The first convolution block has 64 filters, and the second convolution block has 128 filters.
• The final configurations of the model architectures were obtained through a grid search of different hyperparameters and layer configurations.

3.3. Training process

• During training, the authors intentionally excluded this time-dependence from the model.
• This was done (1) using the SDOML dataset, which has already been corrected for degradation effects, (2) without assuming any relation between t and α and avoiding to use t as an input feature, and (3) temporally shuffling the data used for training.
• The authors did not use the full dataset to calculate the gradient descent and propagated back to update the network parameters or weights in the minibatch concept.
• This procedure allowed decreasing the computation cost while still obtaining a lower variance.
• The test dataset, that is, the sample of data the authors used to provide an unbiased evaluation of a model fit on the training dataset, holds images obtained during August to October between 2010 and 2013, again every six hours per day, totaling 9422 images over 1346 time stamps.

3.4. Toy model formulation to probe the multichannel relation

• Using the described CNN model, the authors tested the hypothesis using a toy dataset, which is simpler than the SDOML dataset.
• The authors tested whether the physical relation between the morphology and brightness of solar structures (e.g., ARs, coronal holes) across multiple AIA channels would help the model prediction.
• This toy dataset was designed so that the authors were able to independently test the effect of the presence of (a) a relation between brightness Ai and size σ, and (b) a relation between Ai for various channels; and the presence of both (a) and (b) on the performance.
• The bottom left cell shows the loss when there is no cross-channel correlation, but it has a relation between brightness and size.
• Ai and size σ were removed, the performance was poorer, which increased the MSE loss.

3.5. Reconstruction of the degradation curve using the CNN models

• In order to evaluate the model on a different dataset from the dataset the authors used in the training process, they used both singleand multichannel CNN architectures to recover the instrumental degradation in the entire period of SDO (from 2010 to 2020).
• To produce the degradation curve for the two CNN models, the authors used a dataset that was equivalent to the SDOML dataset, but they did not correct the images for degradation4 (Dos Santos et al. 2021).
• All other preprocessing steps, including masking the solar limb, rescaling the intensity, and so on, remained unchanged.
• A53, page 5 of 12 While the EVE-to-AIA cross-calibration introduced errors of only a few percent (in addition to the calibration uncertainty intrinsic to EVE itself), the FISM-to-AIA cross-calibration errors are considerably larger.
• The feature maps expand their understanding of the model operation.

4. Baseline model

• The authors compared their DNN approach to a baseline motivated by the assumption that the EUV intensity outside magnetically ARs, that is, the quiet Sun, is invariant in time (a similar approach was also considered for the in-flight calibration of some UV instruments, e.g., Schühle et al. 1998).
• It is important to remark that the authors used exactly the same data preprocessing and splitting approach as they used for the neural network model described in Sect. 3.3. From the processed dataset, a set of reference images per channel, Cref , were selected at time t = tref .
• The authors then applied an absolute global threshold value of 5 Mx cm−2 on the coaligned HMI LOS magnetic field maps corresponding to t = tref , such that only those pixels in which BLOS was lower than the threshold were extracted.
• Based on the assumption that the intensity of the quiet-Sun area does not change significantly over time (as discussed in the preceding section), the authors chose to artificially dim these regions by multiplying them with a constant random factor between 0 and 1.
• Subsequently, the dimming factor was obtained by computing the ratio of the two most probable intensity values according to the following equation: αi := Impi Impi,ref .

5.1. Comparing the performances of the baseline model with different CNN architectures

• The results of the learning algorithm were binarized using five different thresholds: the absolute value of 0.05, and relative values of 5%, 10%, 15%, and 20%.
• The authors chose different success rate thresholds to gauge the model, all of which are lower than the uncertainty of the AIA calibration (estimated as ∼10% earlier than May 2014 and ∼28% later).
• When the relative tolerance levels are increased, the mean success rate increases from 27% (for 5% relative tolerance) to 66% (with 20% relative tolerance) and with a 39% success rate in the worst-performing channel (211 Å).
• The most significant improvement was shown in the 94 Å channel with an increase from 32% in the baseline model to about 70% in the single input CNN model, with an absolute tolerance of 0.05.
• The authors stopped the training before epoch 1000; the improvements achieved in the test set over many epochs were only marginal.

5.2. Modeling the channel degradation over time

• In this section the authors discuss the results obtained when the AIA degradation curves V8 and V9 were compared with the singleand multichannel CNN models.
• It is important to note that MEGS-A was used earlier for sounding-rocket calibration purposes, the loss of which caused the V8 and V9 degradation curves to become noisier.
• The different panels of Fig. 6 show that even though the authors trained the single and multichannel models with the SDOML dataset, which was produced and corrected using the V8 degradation curve, the two CNN models predict the degradation curves for each channel quite accurately over time, except for the 94 Å and 211 Å channel.
• The deviations of the predicted values for these two channels fall well within the shaded area associated with the V9 calibration curve.
• The first is the two-sample Kolmogorov-Smirnov test (KS), which determines whether two samples come from the same distribution (Massey 1951), and the null hypothesis assumes that the two distributions are identical.

5.3. Feature maps

• As mentioned in Sect. 3.5, the feature maps are the result of applying the filters to an input image.
• That is, at each layer, the feature map is the output of that layer.
• In Fig. 7 the authors present such maps obtained from the output of the last convolutional layer of their CNN.
• The top row shows the reference input image observed at 193 Å the authors used in this analysis, with its intensity scaled between 0 − 1 pixel units, and the bottom row shows 4 representative feature maps (out of a total of 128) with their corresponding weights.
• The predicted α dimming factors from the model are given by the sigmoid activation function applied to a linear combination of these features.

6. Concluding remarks

• This paper reports a novel ML-based approach to autocalibration and advances their understanding of the cross-channel relation of different EUV channels by introducing a robust novel method to correct for the EUV instrument time degradation.
• The authors also found that the CNN models reproduce the most recent sounding-rocket-based degradation curves (V8 and V9) very closely and within their uncertainty levels.
• This paper finally presents a unique possibility of autocalibrating deep-space instruments such as those onboard the STEREO spacecraft and the recently launched remote-sensing instrument called Extreme Ultraviolet Imager (Rochus et al. 2020) on board the Solar Orbiter satellite (Müller et al. 2020), which are too far away from Earth to be calibrated using a traditional method such as sounding-rockets.
• This project was partially conducted during the 2019 Frontier Development Lab (FDL) program, a co-operative agreement between NASA and the SETI Institute.
• L.F.G.S was supported by the National Science Foundation under Grant No. AGS-1433086.

Figures

Fig. 4. Histograms of the pixel values for the 304 Å channel. In blue we show the histogram for the refence image, and in red the histogram for the dimmed image. The y-axis is the number of pixels, and the x-axis is the pixel intensity [DN/px/s]. The modes are marked with blue and red lines for the reference and dimmed images, respectively.

Fig. 1. Set of images to exemplify how degradation affects the AIA channels. The two sets are composed of seven images from different EUV channels. From left to right: AIA 94 Å, AIA 131 Å, AIA 171 Å, AIA 193 Å, AIA 211 Å, AIA 304 Å, and AIA 335 Å. Top row: images from 13 May 2010, and bottom row: images from 31 August 2019, without correction for degradation. The 304 Å channel images are in log-scale because the degradation is severe.

Table 2. Results of the baseline and CNN models applied to all the EUV AIA channels.

Table 3. Goodness-of-fit metrics for single- and multichannel models with reference to the V9 degradation curve.

Fig. 7. Feature maps obtained from the last layer of the CNN of our model. Top row: a sample input in the AIA 193 Å channel, and bottom row: 4 representative feature maps out of 128 different feature maps from the final convolutional layer of the multichannel CNN model.

Fig. 5. Graphic of the evolution of training and testing MSE loss through the epochs.

Fig. 3. CNN architectures. The single-channel architecture with a single wavelength input that is composed of two blocks of a convolutional layer is shown at the top, with the ReLU activation function and max pooling layer, followed by a fully connected (FC) layer and a final sigmoid activation function. The multichannel architecture with a multiwavelength input that is composed of two blocks of a convolutional layer is shown at the bottom, with the ReLU activation function and max pooling layer, followed by an FC layer and a final sigmoid activation function. Figures constructed following Iqbal (2018).

Fig. 2. Colocated set of images of the seven EUV channels of AIA to exemplify structures that are observed at different wavelengths. From left to right: AIA 94 Å, AIA 131 Å, AIA 171 Å, AIA 193 Å, AIA 211 Å, AIA 304 Å, and AIA 335 Å.

Table B.1. Detailed single-channel architecture.

Table B.2. Detailed multichannel architecture.

Table A.1. AIA channel scaling units.

Fig. 6. Channel degradation over time. From top to bottom: Channel 94 Å (blue), 131 Å (yellow), 171Å (green), 193 Å (red), 211 Å (purple), 304 Å (brown), and 335 Å (magenta). The solid black (gray) curve is the degradation profile of AIA calibration release V9 (V8). The gray shaded area correspond to the error (10% earlier than May 2014 and 28% error later) of the degradation curve V9. The colored shaded areas are the standard deviation of the CNN models. The vertical dashed black line is the last available observation from EVE MEGS-A data, and the vertical dashed gray line is the last training date.

Table 1. MSE for all combinations proposed in Sect. 3.4.

Full text

A&A 648, A53 (2021)
https://doi.org/10.1051/0004-6361/202040051
c
 ESO 2021
Astronomy
&
Astrophysics
Multichannel autocalibration for the Atmospheric Imaging
Assembly using machine learning
Luiz F. G. Dos Santos
1,2
, Souvik Bose
3,4
, Valentina Salvatelli
5,6
, Brad Neuberg
5,6
, Mark C. M. Cheung
7
,
Miho Janvier
8
, Meng Jin
6,7
, Yarin Gal
9
, Paul Boerner
7
, and Atılım Güne¸s Baydin
10,11
1
Heliophysics Science Division, NASA, Goddard Space Flight Center, Greenbelt, MD 20771, USA
2
The Catholic University of America, Washington, DC 20064, USA
e-mail: 51guedesdossantos@cua.edu
3
Rosseland Center for Solar Physics, University of Oslo, PO Box 1029 Blindern, 0315 Oslo, Norway
4
Institute of Theoretical Astrophysics, University of Oslo, PO Box 1029 Blindern, 0315 Oslo, Norway
5
Frontier Development Lab, Mountain View, CA 94043, USA
6
SETI Institute, Mountain View, CA 94043, USA
7
Lockheed Martin Solar & Astrophysics Laboratory (LMSAL), Palo Alto, CA 94304, USA
8
Université Paris-Saclay, CNRS, Institut d’astrophysique spatiale, Orsay, France
9
OATML, Department of Computer Science, University of Oxford, Oxford OX1 3QD, UK
10
Department of Computer Science, University of Oxford, Oxford OX1 3QD, UK
11
Department of Engineering Science, University of Oxford, Oxford OX1 3PJ, UK
Received 2 December 2020 / Accepted 29 January 2021
ABSTRACT
Context. Solar activity plays a quintessential role in affecting the interplanetary medium and space weather around Earth. Remote-
sensing instruments on board heliophysics space missions provide a pool of information about solar activity by measuring the solar
magnetic field and the emission of light from the multilayered, multithermal, and dynamic solar atmosphere. Extreme-UV (EUV)
wavelength observations from space help in understanding the subtleties of the outer layers of the Sun, that is, the chromosphere
and the corona. Unfortunately, instruments such as the Atmospheric Imaging Assembly (AIA) on board the NASA Solar Dynamics
Observatory (SDO), suffer from time-dependent degradation that reduces their sensitivity. The current best calibration techniques rely
on flights of sounding rockets to maintain absolute calibration. These flights are infrequent, complex, and limited to a single vantage
point, however.
Aims. We aim to develop a novel method based on machine learning (ML) that exploits spatial patterns on the solar surface across
multiwavelength observations to autocalibrate the instrument degradation.
Methods. We established two convolutional neural network (CNN) architectures that take either single-channel or multichannel input
and trained the models using the SDOML dataset. The dataset was further augmented by randomly degrading images at each epoch,
with the training dataset spanning nonoverlapping months with the test dataset. We also developed a non-ML baseline model to assess
the gain of the CNN models. With the best trained models, we reconstructed the AIA multichannel degradation curves of 2010–2020
and compared them with the degradation curves based on sounding-rocket data.
Results. Our results indicate that the CNN-based models significantly outperform the non-ML baseline model in calibrating instru-
ment degradation. Moreover, multichannel CNN outperforms the single-channel CNN, which suggests that cross-channel relations
between different EUV channels are important to recover the degradation profiles. The CNN-based models reproduce the degradation
corrections derived from the sounding-rocket cross-calibration measurements within the experimental measurement uncertainty, indi-
cating that it performs equally well as current techniques.
Conclusions. Our approach establishes the framework for a novel technique based on CNNs to calibrate EUV instruments. We
envision that this technique can be adapted to other imaging or spectral instruments operating at other wavelengths.
Key words. Sun: UV radiation – techniques: image processing – telescopes – Sun: activity – methods: data analysis
1. Introduction
Solar activity plays a significant role in affecting the inter-
planetary medium and space weather around Earth and all the
other planets of the Solar System (Schwenn 2006). Remote-
sensing instruments on board heliophysics missions can pro-
vide a wealth of information about solar activity, primarily by
capturing the emission of light from the multilayered solar atmo-
sphere, thereby leading to the inference of various physical quan-
tities such as magnetic fields, plasma velocities, temperature, and
emission measure.
NASA currently manages the Heliophysics System Obser-
vatory (HSO), which consists of a group of satellites that con-
stantly monitor the Sun, its extended atmosphere, and the space
environments around Earth and other planets of the Solar Sys-
tem (Clarke 2016). One of the flagship missions of HSO is
the Solar Dynamics Observatory (SDO, Pesnell et al. 2012).
Launched in 2010, SDO has been instrumental in monitoring
solar activity and providing a high volume of valuable scien-
tific data every day with a high temporal and spatial resolu-
tion. It has three instruments on board: the Atmospheric Imaging
Assembly (AIA, Lemen et al. 2012), which records images with
Article published by EDP Sciences A53, page 1 of 12

A&A 648, A53 (2021)
May 13
th
, 2010
094
Å
131
Å
171
Å
193
Å
211
Å
304
Å
335
Å
August 31
st
, 2019
Fig. 1. Set of images to exemplify how degradation affects the AIA channels. The two sets are composed of seven images from different EUV
channels. From left to right: AIA 94 Å, AIA 131 Å, AIA 171 Å, AIA 193 Å, AIA 211 Å, AIA 304 Å, and AIA 335 Å. Top row: images from 13
May 2010, and bottom row: images from 31 August 2019, without correction for degradation. The 304 Å channel images are in log-scale because
the degradation is severe.
high spatial and temporal resolution of the Sun in the ultravi-
olet (UV) and extreme UV (EUV); the Helioseismic and Mag-
netic Imager (HMI, Schou et al. 2012), which provides maps of
the photospheric magnetic field, solar surface velocity, and con-
tinuum filtergrams; and the EUV Variability Experiment (EVE,
Woods et al. 2012), which measures the solar EUV spectral irra-
diance.
Over the past decade, SDO has played a central role in
advancing our understanding of the fundamental plasma pro-
cesses governing the Sun and space weather. This success can
mainly be attributed to its open-data policy and a consistent
high data-rate of approximately two terabytes of scientific data
per day. The large volume of data accumulated over the past
decade (over 12 petabytes) provides a fertile ground for devel-
oping and applying novel machine learning (ML) based data-
processing methods. Recent studies, such as the prediction of
solar flares from HMI vector magnetic fields (Bobra & Couvidat
2015), creation of high-fidelity virtual observations of the
solar corona (Salvatelli et al. 2019 and Cheung et al. 2019),
a forecast of far-side magnetograms from the Solar Terres-
trial Relations Observatory (STEREO, Kaiser et al. 2008), EUV
images (Kim et al. 2019), super-resolution of magnetograms
(Jungbluth et al. 2019), and a map of EUV images from AIA
to spectral irradiance measurements (Szenicer et al. 2019) have
demonstrated the immense potential of ML applications in solar
and heliophysics. In this paper, we use the availability of such
high-quality continuous observations from SDO and apply ML
techniques to address the instrument calibration problem.
One of the crucial issues that limit the diagnostic capabil-
ities of the SDO-AIA mission is the degradation of sensitivity
over time. Sample images from the seven AIA EUV channels
in Fig. 1 show an example of this deterioration. The top row
shows the images observed during the early days of the mis-
sion, from 13 May 2010, and the bottom row shows the cor-
responding images observed more recently on 31 August 2019,
scaled within the same intensity range. The images in the bottom
row clearly appear to be significantly dimmer than their top row
counterparts. In some channels, especially 304 Å and 335 Å the
effect is pronounced.
The dimming effect observed in the channels is due to
the temporal degradation of EUV instruments in space that is
also known to diminish the overall instrument sensitivity with
time (e.g., BenMoussa et al. 2013). The possible causes include
either the outgassing of organic materials in the telescope struc-
ture, which may deposit on the optical elements (Jiao et al.
2019), or the decrease in detector sensitivity due to exposure to
EUV radiation from the Sun.
In general, first-principle models predicting the sensitivity
degradation as functions of time and wavelength are not suffi-
ciently well constrained to maintain the scientific calibration of
these instruments. To circumvent this problem, instrument sci-
entists have traditionally relied on empirical techniques, such as
considering sources with known fluxes, the so-called standard
candles. However, no standard candles exist in the solar atmo-
sphere at these wavelengths because the solar corona is contin-
uously driven and structured by evolving magnetic fields, which
cause localized and intermittent heating. This causes even the
quiet-Sun brightness in the EUV channels to vary significantly,
depending on the configuration of the small-scale magnetic
fields (Shakeri et al. 2015, and the references therein). On the
one hand, the Sun may not be bright enough to appear in the hot-
ter EUV channels such as AIA 94 Å. On the other hand, the EUV
fluxes of active regions (ARs) can vary by several orders of mag-
nitude, depending on whether the AR is in an emerging, flaring,
or decaying state. Moreover, the brightness depends on the com-
plexity of the AR magnetic field (van Driel-Gesztelyi & Green
2015). Finally, ARs in the solar corona can evolve on timescales
ranging from a few minutes to several hours, leading to obvious
difficulties in obtaining a standard flux for the purpose of cali-
bration.
The current state-of-art methods for compensating for this
degradation rely on cross-calibration between AIA and EVE
instruments. The calibrated measurement of the full-disk solar
spectral irradiance from EVE is passed through the AIA wave-
length (filter) response function to predict the integrated AIA
signal over the full field of view. The predicted band irradiance is
later compared with the actual AIA observations (Boerner et al.
2014). The absolute calibration of SDO-EVE is maintained
through periodic sounding-rocket experiments (Wieman et al.
2016) that use a near-replica of the instrument on board SDO
to gather a calibrated observation that spans the short interval
of the suborbital flight (lasting a few minutes). A comparison
of the sounding-rocket observation with the satellite instrument
observation provides an updated calibration, revealing long-term
trends in the sensitivities of EVE and thus of AIA.
Sounding rockets are undoubtedly crucial; however, the
sparse temporal coverage (there are flights roughly every two
years) and the complexities of intercalibration are also potential
A53, page 2 of 12

L. F. G. Dos Santos et al.: Multichannel autocalibration for the Atmospheric Imaging Assembly using machine learning
sources of uncertainty in the interinstrument calibration. More-
over, the intercalibration analysis has long latencies of months
and sometimes years between the flights and depending on the
times at which the calibration can be updated based on the data
analysis of the data obtained during the flight; this type of cali-
bration is also limited to observations from Earth and thus can-
not easily be used to calibrate missions in deep space (e.g.,
STEREO).
In this paper, we focus on automating the correction of
the sensitivity degradation of different AIA wavebands by
exclusively using AIA information and adopting a deep neu-
ral network (DNN, Goodfellow et al. 2006) approach, which
exploits the spatial patterns and cross-spectral correlations of the
observed solar features in multiwavelength observations of AIA.
We compare our approach with a non-ML method motivated by
solar physics heuristics, which we call the baseline model. We
evaluate the predicted degradation curves with those obtained
through the sounding rocket cross-calibration described above.
To the best of our knowledge, this is the first attempt to develop
a calibration method of this type
1
. The approach developed in
this work may be able to remove a major impediment for devel-
oping future HSO missions that can deliver solar observations
from different vantage points beyond the Earth orbit.
The paper is structured as follows: in Sect. 2 we present and
describe our dataset. In Sect. 3 we illustrate the technique and
how it has been developed. In Sect. 3.1 we state the hypothe-
sis and propose a formulation of the problem, in Sect. 3.2 we
present the CNN models, in Sect. 3.3 we describe the training
process and the evaluation, in Sect. 3.4 we probe the multichan-
nel relation, and in Sect. 3.5 we reconstruct the temporal degra-
dation curve. Furthermore, in Sect. 4 we present the baseline,
followed by Sect. 5, in which we present and discuss the results.
Concluding remarks are given in Sect. 6.
2. Data description and preprocessing
We used the preprocessed SDO-AIA dataset from Galvez et al.
(2019, hereafter referred to as SDOML). This dataset is ML-
ready to be used for any kind of application related to the AIA
and HMI data, and it consists of a subset of the original SDO
data that covers from 2010 to 2018. It comprises the seven EUV
channels, two UV channels from AIA, and vector magnetograms
from HMI. The data from the two SDO instruments are tem-
porally aligned, with cadences of 6 min for AIA (instead of the
original 12 s) and EVE and 12 min for HMI. The full-disk images
are downsampled from 4096×4096 to 512× 512 pixels and have
an identical spatial sampling of ∼ 4
00
. 8 per pixel.
In SDOML, the AIA images are compensated for the expo-
sure time and corrected for instrumental degradation over time
using piecewise-linear fits to the V8 corrections released by the
AIA team in November 2017
2
. These corrections are based on
cross-calibration with SDO-EVE, where the EVE calibration is
maintained by periodic sounding rocket under flight (including,
1
We presented an early-stage result of this work as an extended
abstract at the NeurIPS workshop on ML and Physical Sciences 2019
(which has no formal proceedings) (NeurIPS 2019, Neuberg et al.
2019), where we described some preliminary results in this direction.
In this paper, we extend the abstract with full analyses and discussion
of several important issues, such as the performance on the real degra-
dation curve and the limitations of the presented models, which are both
crucial for evaluating the applicability of this ML-based technique.
2
Available at https://aiapy.readthedocs.io/en/stable/
generated/gallery/instrument_degradation.html#
sphx-glr-generated-gallery-instrument-degradation-py
in the case of the V8 corrections, a flight on 1 June 2016). Con-
sequently, the resulting dataset offers images where changes in
pixel brightness are directly related to the state of the Sun rather
than instrument performance.
We applied a few additional preprocessing steps. First, we
downsampled the SDOML dataset to 256 × 256 pixels from
512 × 512 pixels. We established that 256 × 256 is a sufficient
resolution for the predictive task of interest (inference of a single
coefficient), and the reduced size enabled quicker processing and
more efficient use of the computational resources. Second, we
masked the off-limb signal (r > R

) to avoid possible contam-
ination due to the telescope vignetting. Finally, we rescaled the
brightness intensity of each AIA channel by dividing the image
intensity by a channel-wise constant factor. These factors repre-
sent the approximate average AIA data counts in each channel
and in the period from 2011 to 2018 (derived from Galvez et al.
2019). This rescaling is implemented to set the mean pixel val-
ues close to unity in order to improve the numerical stability and
the training convergence of the CNN. Data normalization such
as this is standard practice in NNs (Goodfellow et al. 2006). The
specific values for each channel are reported in Appendix A.
3. Method
3.1. Formulation of the problem
It is known that some bright structures in the Sun are observed at
different wavelengths. Figure 2 shows a good example from 07
April 2015 of a bright structure in the center of all seven EUV
channels from AIA. Based on this cross-channel structure, we
established a hypothesis divided into two parts. First, that the
morphological features and the brightness of solar structures in
a single channel are related (e.g., typically, dense and hot loops
over ARs). Second, that this relation of the morphological fea-
tures and the brightness of solar structures can be found in mul-
tiple AIA channels. We hypothesize that these two relations can
be used to estimate the dimming factors, and that a deep-learning
model can automatically learn these inter- and cross-channel pat-
terns and exploit them to accurately predict the dimming factor
of each channel.
To test our hypothesis, we considered a vector C =
{C
i
, i ∈ [1, . . . , n]} of multichannel synchronous SDO/AIA
images, where C
i
denotes the i-th channel image in the vector,
and a vector α = {α
i
, i ∈ [1, . . . , n]}, where α
i
is the dimming
factor that is independently sampled from the continuous uni-
form distribution between [0.01, 1.0]. We chose an upper bound
value of α
i
= 1 because we only considered dimming of the
images and not enhancements. Furthermore, we created a corre-
sponding vector of dimmed images as D = {α
i
C
i
, i ∈ [1, . . . , n]},
where D is the corresponding dimmed vector. It is also to be
noted that the dimming factors α
i
were applied uniformly per
channel and are not spatially dependent. The spatial dependence
of the degradation is assumed to be accounted for by regularly
updated flat-field corrections applied to AIA images. Our goal
in this paper is to find a deep learning model M : D → α that
retrieves the vector of multichannel dimming factors α from the
observed SDO-AIA vector D.
3.2. Convolutional neural network model
Deep learning is a highly active subfield of ML that focuses on
specific models called deep neural networks (DNNs). A DNN is
a composition of multiple layers of linear transformations and
nonlinear element-wise functions (Goodfellow et al. 2006). One
A53, page 3 of 12

A&A 648, A53 (2021)
-200.0"-100.0" 0.0" 100.0"
0.0"
-50.0"
-100.0"
-150.0"
-200.0"
Solar X (arcsec)
Solar Y (arcsec)
2015-04-07T22:32:49.120
AIA 94 AIA 131 AIA 171 AIA 193 AIA 211 AIA 304 AIA 335
Fig. 2. Colocated set of images of the seven EUV channels of AIA to exemplify structures that are observed at different wavelengths. From left to
right: AIA 94 Å, AIA 131 Å, AIA 171 Å, AIA 193 Å, AIA 211 Å, AIA 304 Å, and AIA 335 Å.
Fig. 3. CNN architectures. The single-channel architecture with a sin-
gle wavelength input that is composed of two blocks of a convolutional
layer is shown at the top, with the ReLU activation function and max
pooling layer, followed by a fully connected (FC) layer and a final sig-
moid activation function. The multichannel architecture with a multi-
wavelength input that is composed of two blocks of a convolutional
layer is shown at the bottom, with the ReLU activation function and
max pooling layer, followed by an FC layer and a final sigmoid activa-
tion function. Figures constructed following Iqbal (2018).
of the main advantages of deep learning is that it can learn the
best feature representation for a given task from the data without
the need to manually engineer these features. DNNs have pro-
duced the state-of-art results in many complex tasks, including
object detection in images (He et al. 2016), speech recognition
(Amodei et al. 2016) and synthesis (Oord et al. 2016), and trans-
lation between languages (Wu et al. 2016). A DNN expresses a
differentiable function F
θ
: X → Y that can be trained to per-
form complex nonlinear transformations by tuning parameters θ
using gradient-based optimization of a loss function (also known
as objective or error) L(θ) =
P
i
l(F
θ
(x
i
), y
i
) for a given set of
inputs and desired outputs {x
i
, y
i
}.
For the degradation problem summarized in Sect. 3.1, we
considered two CNN architectures (LeCun & Bengio 1995). The
first architecture does not exploit the spatial dependence of mul-
tichannel AIA images and therefore ignores any possible rela-
tion that different AIA channels might have, and it is designed to
explore only the relation in different structures in a single chan-
nel. This architecture is a test of the first hypothesis in Sect. 3.1.
The second architecture is instead designed to exploit possible
cross-channel relations while training, and it tests our second
hypothesis: solar surface features that appear in the different
channels will make a multichannel CNN architecture more effec-
tive than a single-channel CNN that only exploits interchannel
structure correlations. The first model considers a single chan-
nel as input in the form of a tensor with shape 1 × 256 × 256
and has a single degradation factor α as output. The second
model takes in multiple AIA channel images simultaneously as
input with shape n × 256 × 256 and output n degradation factors
α = {α
i
, i ∈ [1, . . . , n]}, where n is the number of channels, as
indicated in Fig. 3.
The single- and multichannel architectures are described in
Fig. 3. They both consist of two blocks of a convolutional layer
followed by a rectified linear unit (ReLU) activation function
(Nair & Hinton 2010) and a max pooling layer. These are fol-
lowed by a fully connected (FC) layer and a final sigmoid acti-
vation function that is used to output the dimming factors. The
first convolution block has 64 filters, and the second convolution
block has 128 filters. In both convolution layers, the kernel size
is 3, meaning that the filters applied on the image are 3 × 3 pix-
els, and the stride is 1, meaning that the kernel slides through
the image one pixel per step. No padding is applied (i.e., no
additional pixels are added at the border of the image to avoid
a change in size). The resulting total learnable parameters (LP)
are 167 809 for the single-channel model and 731 143 for the
multichannel model. The final configurations of the model archi-
tectures were obtained through a grid search of different hyper-
parameters and layer configurations. More details of the archi-
tectures can be found in Appendix B.
We used the open-source software library PyTorch
(Paszke et al. 2017) to implement the training and inference code
for the CNN. The source code that we used to produce this paper
is publicly available
3
.
3.3. Training process
The actual degraded factors α
i
(t) (where t is the time since the
beginning of the SDO mission, and i is the channel) trace a single
trajectory in an n-dimensional space starting with α
i
(t = 0) = 1
∀ i ∈ [1, . . . , n] at the beginning of the mission. During training,
we intentionally excluded this time-dependence from the model.
This was done (1) using the SDOML dataset, which has already
been corrected for degradation effects, (2) without assuming any
relation between t and α and avoiding to use t as an input fea-
ture, and (3) temporally shuffling the data used for training. As
presented in Sect. 3.1, we degraded each set of multichannel
images C by a unique α = {α
i
, i ∈ [1, . . . , n]}. We then devised a
strategy such that from one training epoch to the next, the same
3
Salvatelli et al. (2021) and https://github.com/
vale-salvatelli/sdo-autocal_pub
A53, page 4 of 12

L. F. G. Dos Santos et al.: Multichannel autocalibration for the Atmospheric Imaging Assembly using machine learning
set of multichannel images could be dimmed by a completely
independent set of α dimming factors. This data augmentation
and regularization procedure allows the model to generalize and
perform well in recovering dimming factors over a wide range
of solar conditions.
The training set comprises multichannel images C obtained
during January to July from 2010 to 2013 obtained every six
hours, amounting to a total of 18 970 images in 2710 time
stamps.The model was trained using 64 samples per minibatch,
and the training was performed for 1000 epochs. We did not
use the full dataset to calculate the gradient descent and prop-
agated back to update the network parameters or weights in the
minibatch concept. Instead, we calculated the gradient descent
and corrected the weights while the model was still process-
ing the data. This procedure allowed decreasing the computation
cost while still obtaining a lower variance. As a consequence
of our data augmentation strategy, after 1000 epochs the model
was trained with 2 710 000 unique sets of (input, output) pairs
because we used a different set of α each epoch. We used the
Adam optimizer (Kingma & Ba 2014) in our training with an
initial learning rate of 0.001 and the mean squared error (MSE)
of the predicted degradation factor (α
P
), and the ground-truth
value (α
GT
) was used as the training objective (loss).
The test dataset, that is, the sample of data we used to pro-
vide an unbiased evaluation of a model fit on the training dataset,
holds images obtained during August to October between 2010
and 2013, again every six hours per day, totaling 9422 images
over 1346 time stamps. The split by month between the train-
ing and test data has two objectives: (1) it prevents the bias due
to the variation in the solar cycle, thereby allowing the model
to be deployed in future deep-space missions forecasting α for
future time steps, and (2) it ensures that the same image is never
present in both datasets (any two images adjacent in time will
approximately be the same), leading to a more precise and com-
prehensive evaluation metric.
3.4. Toy model formulation to probe the multichannel relation
Using the described CNN model, we tested the hypothesis using
a toy dataset, which is simpler than the SDOML dataset. We
tested whether the physical relation between the morphology and
brightness of solar structures (e.g., ARs, coronal holes) across
multiple AIA channels would help the model prediction. For this
purpose, we created artificial solar images, in which a 2D Gaus-
sian profile was used (Eq. (1)) to mimic the Sun as an idealized
bright disk with some center-to-limb variation,
C
i
(x, y) = A
i
exp (−[x
2
+ y
2
]σ
−2
), (1)
where A is the amplitude centered at (0, 0), the characteristic
width is σ, and x and y are the coordinates at the image. σ is
sampled from a uniform distribution between 0 and 1. These
images are not meant to be a realistic representation of the Sun.
However, as formulated in Eq. (1), they include two qualities we
posit to be essential for allowing our autocalibration approach to
be effective. The first is the correlation of intensities in the wave-
length channels (i.e., ARs tend to be bright in multiple channels).
The second is the existence of a relation between the spatial
morphology of EUV structures with their brightness. This toy
dataset was designed so that we were able to independently test
the effect of the presence of (a) a relation between brightness A
i
and size σ, and (b) a relation between A
i
for various channels;
and the presence of both (a) and (b) on the performance. To eval-
uate this test, we used the MSE loss and expect the presence of
both (a) and (b) to minimize this loss.
Table 1. MSE for all combinations proposed in Sect. 3.4.
Brightness and size
correlation
Yes No
Cross-channel Yes 0.017 0.023
correlation
No 0.027 0.065
Notes. The top left cell shows the scenario of a cross-channel correla-
tion and a relation between brightness and size of the artificial Sun. The
top right cell shows the loss with a cross-channel correlation, but not the
relation between brightness and size. The bottom left cell shows the loss
when there is no cross-channel correlation, but it has a relation between
brightness and size. The bottom right cell presents the loss when the
parameters are chosen freely.
The test result of the multichannel model with artificial solar
images is shown in Table 1. When A
0
∝ σ (linear relation
between size and brightness) and A
i
= A
i
0
(i.e., dependence
across channels; here the superscript i denotes A
0
to the i-th
power), the CNN solution delivered minimum MSE loss (top
left cell). When the interchannel relation (i.e., each A
i
was ran-
domly chosen) or the relation between brightness A
i
and size
σ were removed, the performance was poorer, which increased
the MSE loss. Ultimately, when both A
i
and σ
i
were randomly
sampled for all channels, the model performed equivalently to
randomly guessing or regressing (bottom right cell) and having
the greatest loss of all tests. These experiments confirmed our
hypothesis and indicate that a multichannel input solution out-
performs a single-channel input model in the presence of rela-
tions of the morphology of solar structures and their brightness
across multiple AIA channels.
3.5. Reconstruction of the degradation curve using the CNN
models
In order to evaluate the model on a different dataset from the
dataset we used in the training process, we used both single-
and multichannel CNN architectures to recover the instrumental
degradation in the entire period of SDO (from 2010 to 2020).
To produce the degradation curve for the two CNN models, we
used a dataset that was equivalent to the SDOML dataset, but
we did not correct the images for degradation
4
(Dos Santos et al.
2021). This dataset included data from 2010 to 2020. All other
preprocessing steps, including masking the solar limb, rescal-
ing the intensity, and so on, remained unchanged. The CNN
degradation estimates were then compared to the degradation
estimates obtained from cross-calibration with irradiance mea-
surements that were computed by the AIA team using the tech-
nique described in Boerner et al. (2014).
The cross-calibration degradation curve relies on the daily
ratio of the AIA observed signal to the AIA signal predicted by
SDO-EVE measurements until the end of EVE MEGS-A oper-
ations in May 2014. From May 2014 onward, the ratio is com-
puted using the FISM model (Chamberlin et al. 2020) in place of
the EVE spectra. FISM is tuned to SDO-EVE, so that the degra-
dation derived from FISM agrees with the degradation derived
from EVE through 2014. However, the uncertainty in the correc-
tion derived from FISM is greater than that derived from EVE
observations, primarily because of the reduced spectral resolu-
tion and fidelity of FISM compared to SDO-EVE.
4
The SDOML dataset not corrected for degradation overtime is avail-
able at https://zenodo.org/record/4430801#.X_xuPOlKhmE
A53, page 5 of 12

Frequently asked questions

Q1. What are the contributions mentioned in the paper "Multichannel autocalibration for the atmospheric imaging assembly using machine learning" ?

The authors aim to develop a novel method based on machine learning ( ML ) that exploits spatial patterns on the solar surface across multiwavelength observations to autocalibrate the instrument degradation. Their approach establishes the framework for a novel technique based on CNNs to calibrate EUV instruments. The dataset was further augmented by randomly degrading images at each epoch, with the training dataset spanning nonoverlapping months with the test dataset. Moreover, multichannel CNN outperforms the single-channel CNN, which suggests that cross-channel relations between different EUV channels are important to recover the degradation profiles. The authors envision that this technique can be adapted to other imaging or spectral instruments operating at other wavelengths. 

Q2. What are the future works mentioned in the paper "Multichannel autocalibration for the atmospheric imaging assembly using machine learning" ?

The authors showed that the CNNs learned representations that make use of the different features within solar images, but further work needs to be done on this aspect to establish a more detailed interpretation. This paper finally presents a unique possibility of autocalibrating deep-space instruments such as those onboard the STEREO spacecraft and the recently launched remote-sensing instrument called Extreme Ultraviolet Imager ( Rochus et al. 2020 ) on board the Solar Orbiter satellite ( Müller et al. 2020 ), which are too far away from Earth to be calibrated using a traditional method such as sounding-rockets. The authors wish to thank IBM for providing computing power through access to the Accelerated Computing Cloud, as well as NASA, Google Cloud and Lockheed Martin for supporting this project. This is particularly promising, given that no time information has been used in training the models.