Deep Learning Technology for Predicting Solar Flares from (Geostationary Operational Environmental Satellite) Data

—Solar activity, particularly solar flares can have significant detrimental effects on both space-borne and grounds based systems and industries leading to subsequent impacts on our lives. As a consequence, there is much current interest in creating systems which can make accurate solar flare predictions. This paper aims to develop a novel framework to predict solar flares by making use of the Geostationary Operational Environmental Satellite (GOES) X-ray flux 1-minute time series data. This data is fed to three integrated neural networks to deliver these predictions. The first neural network (NN) is used to convert GOES X-ray flux 1-minute data to Markov Transition Field (MTF) images. The second neural network uses an unsupervised feature learning algorithm to learn the MTF image features. The third neural network uses both the learned features and the MTF images, which are then processed using a Deep Convolutional Neural Network to generate the flares predictions. To the best of our knowledge, this work is the first flare prediction system that is based entirely on the analysis of pre-flare GOES X-ray flux data. The results are evaluated using several performance measurement criteria that are presented in this paper.


I. INTRODUCTION
The concept of space weather has been defined by the US National Space Weather Program as "Conditions on the Sun and in the solar wind, magnetosphere, ionosphere and thermosphere that can influence the performance and reliability of space-borne and ground-based technological systems and can endanger human life or health" [1].There are several influences, originating from space weather phenomena that detrimentally affect important industries relying on avionics, satellites, mobile communication networks, and electricity distribution [2].All these industries touch our daily lives and this means that space weather can impact our lives dramatically.
Painstaking efforts are currently being made in a number of international centres to create accurate solar flare prediction systems.This is because many infrastructures could be affected by significant flares and the cost of building an accurate solar flare prediction system would be much cheaper than the cost of repairing damage caused by such a flare.In this work, the proposed prediction system generates two probabilities for Event and No-event.Event predictions cover significant X and M class flares that might be harmful, while No-event predictions cover no-flares and the non-harmful A, B and C class flares.
Although scientific progress has increased enormously the rate of generation of data monitoring solar activity, scientists are not yet able to fully understand all the detailed causes of solar flares.Consequently, efforts are being made to develop methods to predict solar storms, making direct use of the data using advances in data analysis.
Since 1987, there have been many approaches that attempted to predict solar flares.The first solar flare prediction system (called THEOPHRASTUS) was launched by the Space Environment Services Centre at NOAA, and it predicts X-ray flares with a time window of 24 hours [3].More recently, three solar flare prediction systems, ASSA (Automatic Solar Synoptic Analyser), MAG4 (Magnetic Forecast system) [7] and ASAP (Automated Solar Activity Prediction), have become a part of the NASA Integrated Space Weather Analysis (ISWA) system [5] and these three systems are briefly described below.
The first system, ASSA, is based on an artificial neural network technique and the ASSA coronal hole data archive, from the period 1997 till 2013, including SDO solar images, to predict solar flares, solar radiation storms and geomagnetic storms.ASSA predicts C, M and X flares.ASSA predictions are based on statistical analysis of the ASSA sunspot catalogue [6].The second system, MAG4 was developed at the University of Alabama in Huntsville, to assist NASA Space Radiation Analysis Group (SRAG) at the Johnson Space Flight Centre.MAG4 is using Magnetogram data for the Sun.MAG4 forecasts X and M class flares, CMEs, and Solar Proton Events (SPE) using McIntosh active-region (AR) classes as the basis of their forecasts [7].The University of Bradford developed a forecasting model, the Automated Solar Activity Prediction (ASAP) system in 2009.ASAP uses McIntosh classes and other sunspots features which it generates from the solar data.www.ijacsa.thesai.orgASAP uses SDO/HMI Continuum and Magnetogram images as an input to the system, also it uses two neural networks to predict solar flares [3].
Recently, the new field of deep learning neural network research has achieved remarkable successes compared with previous artificial intelligence methods [5].These include complex tasks like medical diagnoses, dealing with huge amounts of data, pattern recognition and numerous others, such as the virtualization frameworks for big data reported in [8].Using the deep learning technology for space weather prediction is still a novel area of research, which needs to be investigated to help analyse the huge amount of solar activity data that are publically available.

UFCORIN (Universal Forecast Constructor by Optimized Regression of Inputs
) is open-source software available online which has been used to predict general time series and solar flares.This system uses HMI image data and GOSE X-ray data as input to predict X, M, and C solar flare class.In 2016, UFCORIN was extended to use deep learning, and provides 24-hour-ahead predictions of solar flares, every 12 minutes by using a deep learning approach.
In this paper, we introduce a solar flare prediction system, summarised in the following subsection, working solely with GOES X-ray flux data that integrates three neural networks to deliver these predictions and provides an automated prediction of solar flares by utilising deep learning techniques.
GOES data are available in real-time (available every minute) and they provide a general indication of flaring across the solar disk.These data come in soft and hard x-ray and are available from 2002.However, GOES data provide an indication of flaring without much info about the exact location of flaring on the solar disk.This could be one of the reasons why it is not used heavily for space weather prediction.The format of GOES Data is also challenging as it is represented as a time-series signal, which makes it challenging for machinelearning based prediction (Deep learning in particular).

A. Overview of the System
Fig. 1 shows the system model which consists of three units.Starting from the input (GOES X-ray flux time series data) to the output (solar flare prediction) and including the evaluation of the predictive performance.Unit 1in Fig. 1 converts a sequence of GOES X-ray flux 1minute data time series data to a 64 × 64 MTF image in two stages.Firstly, it converts the original text data to a Markov Transition Matrix.Then it encodes the Markov Transition Matrix as a 64 × 64 Markov Transition Field (MTF) image as illustrated in Fig. 6.Unit 2 in Fig. 1 learns the features within the MTF images.Unit 2 pre-processes and normalizes the images and then divides the 64×64 images into 64 8×8 patches.These patches are encoded using a Back-propagation Autoencoder to obtain learned feature mappings as indicated in Fig. 1.Unit 3 in Fig. 1 provides predictions for solar flares using a CNN.This unit starts by utilising the historical knowledge and linking the MFT images with the Flare or No-Flare labels.Subsequently, datasets are created for training and testing the neural networks.After training on the associated dataset is carried out, the trained CNN is run on the test dataset to generate prediction results, which are evaluated using space weather verification metrics.
The rest of this paper is organized as follows.Section 2 describes the operation of Unit 1 which converts GOES X-ray flux time series data to 64 ×64 MTF images.Section 3 describes Unit 2, which learns features within MTF images using an unsupervised learning algorithm by applying backpropagation.Section 4 describes Unit 3, which makes solar flare predictions using a Deep Convolutional Neural Network.Section 5 discusses the evaluation and performance of the whole system and Section 6 presents concluding remarks and suggestions for future work.

A. The Source X-Ray Data
In this work, 1-minute X-ray flux data from the American Geostationary Operational Environmental Satellites (GOES) are used.The data used are provided from four GOES satellites, GOES-10, GOES-11, GOES-14, and GOES-15.All the data produced are archived and available, and it can be found online at [9].Two X-ray channels are available as shown in Fig. 2; a harder X-ray channel (0.05-0.4 nm), and a softer Xray channel (0.1-0.8 nm) [10].For this work, the soft channel is used because provides information about the intensity of solar flares and is used in this work to investigate its suitability for investigating the temporal evolution of flares [10].

B. Extraction of Relevant X-Ray Flux Data
The temporal evolution of solar flares generally occurs in three phases [4].
 Pre-flare phase: This is the region shown in Fig. 3 which consists of fluctuations and a slow increase of Xray flux before the start of the flare event.
 Impulsive phase: Here the X-ray flux increase quickly and the main flare energy release occurs during this phase.
 Gradual phase: In this phase, the X-ray flux gradually decreases to the background level.
Fig. 4 shows the cropped AIA images of a flaring region corresponding to the GOES X-ray data regions in Fig. 3.The left image in Fig. 4, captured in the pre-flare phase, shows two sets of nested loops.The middle image in Fig. 4, captured during the main phase, shows inner loops becoming significantly brighter.In the right-hand image, the flare launches a CME.There are many relationships which have been recognized between the pre-flare activities and flaring, and these appear as loop brightening activities [15].However, the method introduced here bases its prediction solely on changes in the overall X-ray flux during the pre-flare phase.

C. Prediction Optimization for Different Time Windows
The Time windows of 20, 30, 60 and 120 minutes between the end of a data sample and the start of a flare/no-flare are investigated, using the Quadratic score QR, to determine the time window with the best prediction performance.QR is widely used as a verification measure to evaluate the accuracy of prediction.The prediction accuracy is calculated by finding the mean square error between the predictions and the observations as given by [2].
where o t are the binary observation outcomes where 1 means that flare occurred and 0 means that a flare did not happen, N is the sample size, and f t is the prediction probability.QR ranges from 0 (perfect prediction) to 1 (worst possible prediction) [18].
The result for each time window is shown in Table I.It is clearly seen that the best QR is when the time window equals 20 minutes.To find the prediction window duration that would provide the best QR value, we followed the method presented in [2] and applied QR to determine the best prediction window duration.

D. Data Presentation
Fig. 5 shows a sub-system that has been created to generate datasets by selecting specific data from GOES X-ray flux 1minute data using three steps.The first step identifies a flare.Then selects 120 minutes of data, starting 140 minutes before the beginning of the flaring event.Finally, the selected data is saved in a matrix as described in the next subsection.

E. Conversion of Time Series Data to MTF Images
Temporal and frequency correlations are major dependencies embedded in time series data.To build a comprehensive but intuitive visualization, the extracted features of the designed data transformation framework should be able to represent the dynamics in both time and frequency while there should exist a reverse operation to map the information back to the raw GOES time series.The following sub-sections describe how to encode the dynamical frequency information in the temporal ordering, illustrated in Fig. 6, step by step.
The main idea of this stage is to use GOES time series data to generate Markov transition field while maintaining the timeseries properties.The method applied in this research is taken from [14].MTF images were generated by applying the code used in [14] to GOES data.

III. LEARNING THE FEATURES WITHIN MTF IMAGES
The Auto-encoder is an unsupervised back-propagation neural network which tries to learn a function hW, b (x) ≈ x, and is adjusted so that the input values correspond to the target y (i)  ̂(i) [12].In this work, we assume x is the input corresponding to the pixel intensity values for an 8×8 MTF image patch with 64 pixels so x = 64, and there are s 2 = 32 hidden units in layer L 2 .The network is required to learn a compressed representation of the input, because there exist only 32 hidden units.Therefore the auto-encoder should attempt to reconstruct the input to 8 × 8 images (64 pixels) [16] as illustrated in Fig. 7.

IV. PREDICTION OF SOLAR FLARES USING A DEEP CONVOLUTIONAL NEURAL NETWORK
As you can see in Fig. 8 the Convolutional Neural Network (CNN) consists of convolutional layers and sub-sampling layers followed by fully connected layers.

A. The Convolutional Layer
The input to this layer is a d × d × ch MTF image where d is the height and the width of the image (d =64 in this case) and ch is the number of channels.Since the MTF images are RGB images, ch = 3.As illustrated in Fig. 9 the convolutional layer uses Kf filters (also called Kernels) of size n × n × ch where n is the dimension of the filter and n =8 to produce feature maps.The Kf filters are convolved over the MTF image to create Kf feature maps of size d−n+1 [16].

B. The Pooling Layer
After the generation of the feature maps by the convolutional layer, the features are then used for classification.Fig. 10 shows each feature map is downsampled by max-pooling to size p × p.Typically, p ranges from 2 to 5, for small to big images respectively, and in this work p=4 [16].

C. The Fully Connected Layer
This layer takes the outputs from the previous layers which were reduced to a one-dimensional feature vector.This layer is fully connected and there is just one output for each class label.The high-level inference in the CNN is performed by this fully connected layer.

V. IMPLEMENTATION AND EVALUATION OF THE SYSTEM
Three neural networks are integrated into the system to predict solar flares.Fig. 4 shows the integrated system starting from the input (GOSE data) to the output of the system (Flare/No-Flare prediction).
The first part of the system, which encodes the GOES data to MTF images, is implemented in Python and the rest of the system is implemented in Matlab [17].The system makes flares predictions based on embedded learning rules.The system was trained using training sets covering data from 3rd Dec 2002 till 30th Jan 2017, to ensure this covered a range of activity including both solar Maximum and solar Minimum of the solar cycle.

A. System Evaluation
The performance evaluation was done by comparing the generated predictions with the actual flare occurrences as reported by 1-minute GOES data.The data were taken from four satellites, GOES-10 data covering ( 03

B. Machine Learning using Cross-Validation
Cross-validation is a method that partitions the input data into subsets so that the learning algorithm can be trained on a subset and internally tested on a different subset.Crossvalidation is a useful approach for analysing the prediction performance of machine learning, as it is could help avoid over-fitting.Over-fitting occurs when the learning algorithm performs very well on the training data, but not so well when provided with new data.Different forms of cross-validation exist and the repeated random sub-sampling validation is applied here.This method is based on randomly dividing the data into a number of subsets, which is repeated a number of times so that the learning algorithm is trained and tested on different data.For each repetition, one subset is used for training and the rest are used to evaluate the prediction performance by calculating a number of forecast verification metrics.These measurements are then averaged in order to provide an indication of the effectiveness of the machine learning on the training data [19].
Two separate portions of data are created: a training portion (60%) and a testing portion (40%).The MTF images and their corresponding flare/no-flare classifications from the training portion are fed into the learning algorithm for training purposes.When the training process is completed, the learning algorithm is fed with the MTF images from the testing portion.The learning algorithm attempts to predict their Flare/No-Flare classifications.These predicted outputs are compared with the testing datasets actual classifications using standard forecast verification measures to evaluate the prediction performance of the learning algorithm.Among the prediction measures, HSS is one of the best indicators of the overall performance of a prediction method since it accounts for correct chance forecasts [20].The cross-validation process is repeated 9 times and the means of the prediction measures are calculated.

C. Verification Results
This system generates a prediction in binary form so 0 means no flare and 1 means a flare.In practice, flares occur rarely compared to no-flares events.Various measures are used to evaluate the predictions of the system.These measures are for categorical prediction (Yes or No) and take the binary prediction as an input to evaluate the output of the system.As shown in Table V, the following four criteria are used to investigate the predictions generated by the system. If an MTF image is associated with a flare, and the system prediction is a flare then this successful prediction is a true positive (TP).
 If an MTF image is associated with a flare, but the system prediction is no-flare then this failed prediction is a false positive (FP).
 If an MTF image is not associated with a flare and the system prediction is no-flare then this successful prediction is a true negative (TN).
 If an MTF image is not associated with a flare and the system prediction is flare then this failed prediction is a false negative (FN).
To further evaluate the results we used various prediction verification measures for the 20 minute time window, shown in Table III.The measures used are the Heidke Skill Score (HSS), the percentage corrects (PC), the false alarm rate (FAR), the probability of detection (POD), and the Brier Score (BS).The formulae for these measures are defined in terms of the abbreviations given in Table IV.
The percentage correct measure, PC, is used to calculate the rate of predictions that are correct [13], and is defined as: (2) The PC rate for the 20 minute time window is shown in Table VI for all the predictions (flare or no-flare) and is 78%.
The Heidke Skill Score (HSS) is a measure showing the improvement of the prediction over random prediction.HSS ranges from -1 (for no correct predictions) to +1 (for very accurate predictions) and a value of zero indicates that the predictions are randomly generated [13].HSS is defined by: (3) HSS is a really useful measure for verifying systems that seek to predict rare events, as in the present case.
The False Alarm Ratio FAR is the fraction of flare predictions that are wrong.The range of FAR is from 0 (best outcome) to 1 (poorest outcome) [18].FAR is defined as: FAR (4) www.ijacsa.thesai.orgThe Probability of Detection (POD) P d , also known as the Hit Rate (H), measures the probability of a solar flare being correctly predicted by the system [18].POD is given by: It ranges from 0 (poorest outcome) to 1 (best outcome).The P d result for this system with a 20 min time window is 0.574.This process separately uses data covering the complete time range (03 Dec 2002 -30 Jan 2017).The prediction measures achieved for datasets are shown in Table VI.It can be seen that the good levels of prediction measures are achieved.

VI. CONCLUSION
This paper has introduced a prediction system that uses a new technology for predicting solar flares from GOES data using deep learning.This is the major contribution of this paper.The system predicts automatically whether a flaring event is going to occur in the next 20 minutes.Different prediction windows were investigated using the QR measure, and the most promising performance was found to be for the 20 minutes prediction window.
The performance of the prediction system introduced here depends on the ability of the deep learning neural network to efficiently classify the MTF images that have been generated to visualise the GOES data.As demonstrated in Table VI all the metrics used to evaluate the prediction performance (POD, FAR, HSS, KSS, and PC) provide fairly good performances.In particular, HSS results prove that the generated predictions are definitely not generated by chance.
The prediction rates for our systems can be improved by exploiting the advanced classification capabilities of machine learning systems.Hence, we believe that it is important to monitor the performance of the system during its initial stages which include comparing the prediction performance with the actual flares reported by NOAA.Evolutionary algorithms may be used to allow the learning algorithms to evolve and provide better optimization.This work is continuing but we believe the initial results, as reported in this paper, are very encouraging.However, we note that not all flares have pre-flare phases occurring before them, and this could be one of the reasons affecting our predictions.To tackle these causes, our system could be integrated with another statistical or machine learning prediction model (e.g.ASAP 1 ).

Fig. 1 .
Fig. 1.The diagram showing the internal procedures of the system.

Fig. 4 .
Fig. 4. Cropped AIA images showing three phases of the solar flare which contributes to the GOES data shown in Fig. 3-From NASA [11]. www.ijacsa.thesai.org

Fig. 5 .
Fig. 5. Creating dataset of a time series of X-ray flux data with a 20-minute data window before the flare occurs.

Fig. 6 .
Fig. 6.Conversion of GOES X-ray data time series data to MTF images.
Dec 2002 -22 Jun 2006) and (11 Apr 2007-30 Dec 2009); GOSE-11 data covering (23 Jun 2006-10 Apr 2007); GOSE-14 data covering (01 Nov 2009 -26 Oct 2010); and finally GOSE-15 data covering (27 Oct 2010 -30 Jan 2017).The number of flaring and No-flaring events for each satellite is detailed in Table II.All GOES X-ray data were taken from [9].As noted earlier in this paper, the data is classified as flaring if they produced at least one M or X class flare in the following 20 min period and No-flare if they did not cause any M or X class flares during that period.To determine the flare prediction capability we carried out experiments with 1-minute GOES data covering (Dec 2002-Dec 2005, Jun 2009-Dec 2012) to train the deep learning algorithm.The data covering (Jun 2006 -Dec 2008, Jun 2013 -Jan 2017) are used to test the system as shown in Table III.Table IV details the number of flare and no-flare data that were used in these experiments.The time coverage of the training set was chosen so that the remaining testing set would contain flare activity from periods around the maximum and minimum levels of solar activity.TABLE II.THE NUMBER OF FLARING AND NO-FLARING FOR GOES-10 DATA COVERING (03 DEC 2002 -22 JUN 2006), (11 APR 2007-30 DEC 2009); GOSE-11 DATA COVERING (23 JUN 2006-10 APR 2007); GOSE-14 DATA COVERING (01 NOV 2009 -26 OCT 2010); GOSE-15 DATA COVERING (27 OCT 2010 -30 JAN 2017) USED IN THIS EXPERIMENT

TABLE I .
THE QUADRATIC SCORE (QR) RESULTS FOR 20, 30, 60 AND 120 MINUTES

TABLE IV .
NUMBER OF FLARE AND NO-FLARE DATA IN TIME INDEPENDENT TRAINING AND TESTING SETS

TABLE VI .
PREDICTION MEASURES ACHIEVED BY APPLYING MACHINE LEARNING AND CROSS-VALIDATION WITH DATASETS COVERING (03 DEC 2002-30 JAN 2017)