Load Forecasting using Autoregressive Integrated Moving Average and Artificial Neural Network

Electric load forecasting is a challenging research problem due to the complicated nature of its dataset involving both linear and nonlinear properties. Various literatures attempted to develop forecasting models that utilized statistical in combination with machine learning approaches deal with the dataset’s linear and nonlinear components to obtain close to accurate predictions. In this paper, autoregressive integrated moving average (ARIMA) and artificial neural networks (ANN) were implemented as forecasting models for a power utility’s dataset in order to predict day-ahead electric load. Electric load data preparation, models implementation and forecasting evaluation was conducted to assess if the prediction of the models met the acceptable error tolerance for day-ahead electric load forecasting. A Java-based system made use of R Statistical Software implemented ARIMA(8,1,2) while Encog Library was used to implement the ANN model composing of Resilient Propagation as the training algorithm and Hyperbolic Tangent as the activation function. The ANN+ARIMA hybrid model was found out to deliver a Mean Absolute Percentage Error (MAPE) of 4.09% which proves to be a viable technique in electric load forecasting while showing better forecasting results than solely using ARIMA and ANN. Through this research, both statistical and machine learning approaches were implemented as a forecasting model combination to solve the linear and non-linear properties of electric load data. Keywords—Electric load forecasting; autoregressive integrated moving average; artificial neural network


I. INTRODUCTION
The fundamental characteristic that makes the electric power industry unique is the product: electricity.A single megawatt, like any other commodity, is frequently bought and resold a number of times before finally being consumed [1]- [3].Load forecasting helps these power utilities make important decisions including decisions on purchasing electric power and load switching.There have been many tools and models used for electric load prediction.Commonly used models include Autoregressive Integrated Moving Average (ARIMA), Artificial Neural Network (ANN), time series and linear regression [2], [3].In practice, hybrid models are being created by combining two models and have been proven to give a more accurate and more precise measure than using the individual models [1], [4]- [6].But even these hybrid models would not always work for every electric load forecasting situation.Similar to non-hybrid models, they still depend on the type of data, the size of the data and the error handling mechanism [4], [5].With this, power utilities would have to choose and ask for experts on recommendations regarding appropriate tools and models to be used in forecasting data.Electric load prediction conducted by these power utilities can be classified into long-term, medium-term, short-term and very short-term forecasting based on the forecasting horizon [3], [7], [8].Short-term load forecasting is mainly used to forecast the day-ahead electric load that is why its accuracy directly affects the economic cost of operators in power utilities and markets [1].Accurate load forecasting is helpful for security, stability, maintenance plans and economic operations in power grids.In order to obtain accurate load prediction, power utilities would need to use a forecasting tool that would work on their data and data structure.ANN, a machine learning tool that is often used for dayahead load forecasting exhibit certain performance characteristics similar to biological neural networks with elements capable of parallel processing like that of the human brain [2], [7], [9].The major advantage of ANN is its flexible nonlinear modeling capability.With ANN, there is no need to specify a particular model form.Rather, the model is adaptively formed based on the features presented from the data.This data-driven approach is suitable for many empirical data sets like electric load where no theoretical guidance is available to suggest an appropriate data generation process [7], [10]- [12].Consequently, ARIMA, popularly known as Box-Jenkins methodology is simple and yields accurate results, exhibiting its wide use by assuming that the future values of a time series have a clear and definite functional relationship with current values, past values and white noise.Although ARIMA models are quite flexible to the extent that they can represent several different types of time series, i.e. pure autoregressive (AR), pure moving average (MA) and combined AR and MA (ARMA) series, their major limitation is the preassumed linear form of the model which means that the ARIMA model has weakness in being able to read non-linear patterns [4], [12], [13].Combining the two models, one which would handle the linearity and another for the non-linearity could give a better output than using just one of them.
A power utility company located in Mindanao, the Philippines has a short-term electric load forecasting system which utilizes linear regression in forecasting electric load.The current linear regression model employed by the existing load forecasting system of the power utility yields forecasted values above the international tolerance error standard of 5%.The www.ijacsa.thesai.orgtechnique used by the power utility company is valid, but there are still different techniques that could provide a better prediction using the electric load data composing of linear and non-linear properties.A hybrid model using ARIMA and ANN would be a viable solution because of its proven efficiency and affectivity in prediction [4], [13].Through data preparation, hybrid model implementation and error measurement evaluation, this study aims to develop a day-ahead electric load forecasting model using ARIMA and ANN.This study hopes to contribute to researches in statistical and machine learning prediction technologies by implementing and evaluating a hybrid short-term electric load forecasting model that could aid power utilities in their decision-making, electric load planning and load power utilization.

A. Electric Load Data Preparation
The dataset used for this study is from the three-year raw monthly electric load data that has been used by the existing system of the power utility from 2012 to 2014.However, only the electric load data coming from three metering points of 28,704 records from December 2013 to October 2014 were utilized since this range is best sufficient to fit an ARIMA and ANN model [1], [12].As shown in Table I    These raw data from a .xlsworksheets were then imported and stored in a PostgreSQL database.The data that was used as inputs in ARIMA modelling was identified as the kilowatt per hour (KW_DEL) column [10].The KW_DEL is the energy delivered from the utility grid which is also the load to maintain and the basis for load prediction [5], [8].Since the load consumption from the .xlsworksheet is recorded per 15 minutes, the maximum load consumption among the four, 15minutes recordings per hour will be set as the load for the hour [10].The electric load data also contains scheduled and unscheduled power interruption records with zero values that could potentially cause the dataset to become out-of-range.To solve this, data correction was then done on the raw electric load data by removing empty or zero values and replacing them with electric load data reflective of the consumed electric load without the power interruption.There is currently no established standardized electric load data correction methodology for power interruptions, but researchers recommend a data correction method by replacing the outlying values with values from the electric load data of the preceding day with the same time frame as with the outlying value [6], [9], [10].
The residuals or the data that was generated from the ARIMA model also underwent a transformation process for the neural network to produce accurate forecasts.In neural networks, it is a best practice to transform input data before use since data transformation makes the training of the network faster and memory efficient resulting for the model to yield accurate forecast results [14].In addition, neural networks only work with data usually between a specified range e.g.-1 to 1 or 0 to 1 [15]- [17].Thus, transformation ensures that data is roughly uniformly distributed between the network inputs and the outputs [17].The transformation technique will use a formula that is the same with the Min-Max normalization process in order for the values to be narrowed down into uniformed variation.Transformation process can be done by using the formula in (1) where z is the transformed value, x is the actual value, and min(x) and max(x) are respectively the minimum and maximum of the dataset.Hence, the transformation technique yielded a value between -1 and 1 and will be used in this study because the residuals have values which are less than 0 making the transformation technique suitable for fitting the data within the unity.

B. ARIMA and ANN Hybrid Model Implementation
In implementing the ARIMA+ANN hybrid model, the actual dataset which is in a database was initially processed in the ARIMA model.As shown in Fig. 1, there are two datasets that can be formulated with the ARIMA forecast: one of which is the linear forecast which by itself is also the ARIMA forecast, the other is the ARIMA residuals which is the difference of the actual dataset and the ARIMA forecast.The linear forecast is stored in a spreadsheet where it will be used later while the ARIMA residuals dataset is being processed in the ANN model.The process of combining the linear and nonlinear values is by adding the values of the same row of the two columns [12].In implementing the ARIMA and ANN models, the data from the database was read in a Java-based system.The ARIMA(8,1,2) model with 8 as the number of autoregressive terms, 1 as the number of non-seasonal differences needed for stationarity and 2 als the number of lagged forecast errors in the prediction equation was integrated into a Java-based system through the use of the R Statistical Software in order to simulate and calculate the linear results.An ANN model with a multilayer perceptron architecture that used Resilient This work is supported by the Mindanao State University-Iligan Institute of Technology (MSU-IIT) as an internally funded research under the Premier Research Institute of Science and Mathematics (PRISM).www.ijacsa.thesai.orgPropagation as the training algorithm and Hyperbolic Tangent as the activation function was also implemented in a Javabased system through the use of Encog Library in order to simulate and calculate the training and testing nonlinear results.Encog is a machine learning framework available for Java, .Net, and C++.Encog supports different learning algorithms such as Bayesian Networks, Hidden Markov Models and Support Vector Machines.However, its main strength lies in its neural network algorithms.
The residuals from the ARIMA model were then implemented in Encog with a maximum error of 0.0001 and a maximum iteration of 10000.As shown in Fig. 2, the ANN implemented in Encog used 1 input layer containing 24 input neurons, 1 hidden layer containing 17 hidden neurons and 1 output layer containing only 1 output neuron was used in making the network for the ANN architecture.The ANN model then delivered the output ANN forecast or the nonlinear forecast.These outputs are equivalent with the predictive size in the ANN process which corresponds to the number of forecast horizon, i.e. 24, covering the 24-hour day ahead forecasting with the 24 hourly values [2], [18].The nonlinear forecast was then placed in the same spreadsheet with the linear forecast where the two values are then added to create an ARIMA+ANN forecast [12].After the ARIMA+ANN hybrid model was implemented, the results from the hybrid model were assessed using error metrics to determine the accuracy.Mean Absolute Percentage Error (MAPE) and Mean Squared Error (MSE) were used as error metrics to quantify the difference between the ARIMA forecast, ANN forecast, ARIMA+ANN forecast and actual electric load.
The dataset used as validation set for the models was the actual consumed electric load data of October 21, 2014.After the predictions were generated by the models, the average MAPE and MSE were then calculated between the three models.Post-transformation of the data involves denormalization or reversing the normalization process [14].The de-transformation process was done by using the same formula used in the transformation process.A graphical representation of the computed results was then generated to illustrate the commonality of the actual and the predicted load values in a much better way.

A. Electric Load Data Preparation Results
The granularity of the raw electric load data was originally per fifteen-minute containing the load consumption for every fifteen minutes that is why the maximum load consumption among the four, per 15-minutes recordings was chosen to reflect the hour's consumption.This is supported by studies which converted their fifteen-minute dataset to an hourly data for the reason that electric dataset are read hourly by spot markets [10], [19].Furthermore, the fifteen-minute data was converted to hourly data by using the maximum load because it is also used by power utility in determining the hourly load.The process of choosing the maximum value from the four, 15minutes records was performed using a Java code created by the researchers.After the maximum load was chosen, it was stored in another table in the database named the hourly table which has the following columns: time which is the combination of both the date and time column from the source table and the corresponding consumed electric load.The final result was a clean one-hour kilowatt delivered data which would serve as inputs to the models.The new number of observation would be 7176 with 7152 observations to be used for training the models and the last 24 observation for testing the models.
The number of observations to be used is just efficient for the models because a larger amount could lead to an over fitted ARIMA model while a smaller amount could lead to an under fitted.According to studies, an ARIMA model is only applicable to a definite and small amount of data [1], [4], [13].The ARIMA model then generated residual data for ANN to process.After the raw date was fed into the ARIMA model, residuals were generated and data were plotted in the graph as shown in Fig. 3.The residual values that were plotted was a random distribution of values with -2680.893454503as the minimum value and 3652.1309335015 as the maximum value.The values are within the range between -2680.893454503 and 3562.1309335015,which means that these values are the boundaries for the upcoming residual transformation.These values were used during the transformation process.II are the actual residuals and normalized residual values which had values ranging from -1 to 1.The residuals had a value of -1, which is the minimum value after transformation for the minimum actual value -2680.893455and 1, which is the maximum value after transformation for the www.ijacsa.thesai.orgmaximum actual value 3652.130934during the transformation.The researchers used Mix-Max transformation technique in transforming the residual values into uniformly distributed numbers between the network inputs and the output.This was supported by studies which used Min-Max transformation in order for the values to narrow down and would be used in ANN training [9], [10].

B. ARIMA and ANN Hybrid Model Implementation Results
The output residuals of the ARIMA forecast was used as inputs to the ANN model.Shown in Fig. 4 are the ANN's forecasted values of the residuals for the next 24-hour data.As observed, the data is in a nonlinear state which also came from the residuals which are initially nonlinear.The ANN forecast data-driven approach is suitable for this kind of empirical data sets where no theoretical guidance is available to suggest an appropriate data generating process [11].In addition neural networks are flexible in terms of nonlinear modeling capability [9], [11], [12].As shown in Table III, these values were then used in the next process as addends to be summed up with the ARIMA forecast.The ARIMA+ANN forecast or the hybrid forecast is the final result of the forecasting process.The results are based on a researcher's assumption that when added, the linear and nonlinear forecast taken from the ARIMA and ANN models would create the final output of the hybrid model [12].To evaluate the forecasting performance, MSE and MAPE were computed for the ARIMA model output, ANN model output and ARIMA+ANN hybrid model output.Table IV shows the MSE of each model.The researchers observed that the ARIMA+ANN hybrid model output had the smallest MSE among the three models having a sum of 0.98, followed by the ARIMA model output with 1.23 and ANN model output of 16.90.The lowest MSE were found in the hours after 13 while the highest MSE are found on 13 and above it.From the result obtained, MSE from each model were below 0.5, and the MSE for both ARIMA and ARIMA+ANN hybrid models are below 0.3.According a study, a good predictive model has a MSE of below 0.5 [20].Thus, the ARIMA model and the ARIMA+ANN model are suitable forecasting model for predicting electric load.
The MAPE of each model result was also calculated in order to evaluate the performance of each output parameter and assess whether the model was able to pass to the acceptable error for electric load forecasting.In calculating the MAPE, the forecasted outputs were subtracted with the actual values and then the difference was divided by the actual values.The results were then multiplied by 100 for percentage.Table V shows the MAPE of the three models.The ARIMA+ANN hybrid model has the lowest MAPE compared to the two other models, making it the best fitting model for the dataset.The MAPE of each model can also be used to check whether the models can be acceptable models for prediction.According to a research, the acceptable MAPE error for testing should be below 15% in order to say that the model is well-performing [21].The ARIMA and ARIMA+ANN hybrid model was able to reach that goal.The MAPE of the ARIMA+ANN hybrid model was also below 5% which is the acceptable error of power utility making it a fitting model for use.The researchers also evaluated the ARIMA+ANN hybrid model if it can still maintain its forecasting accuracy by predicting the 2-days ahead, 3-days ahead, 4-days ahead and 5days ahead and comparing the result to the actual days of October 22 -25, 2014.RMSE and MAPE were used in evaluating the forecasting accuracy of the model.As shown in Table VI, there is an increase in the RMSE and MAPE of the 2-days ahead, 3-days ahead, 4-days ahead and 5-days ahead from the 1-day ahead forecast.The researchers observed that it is not reliable to use the same model for forecasting a number of days ahead.Instead, remodeling should occur every time the model is being used in forecasting the next day.A visualization of the difference between the actual consumed electric load values and the denormalized ARIMA forecasted outputs, ANN forecasted outputs, and ARIMA+ANN hybrid outputs was also generated for evaluation.As shown in Fig. 5, the ARIMA forecasted outputs were compared to the actual values of October 21.Shown in Fig. 6 is the ANN forecasted output as compared to the actual consumed electric load values.The forecasted outputs from ANN had a big difference to the actual data, compared to the ARIMA forecasted outputs.From the hours of 1 to 8, the ANN forecasted data was higher than the actual values, with the exception of hour 2 which is close to the actual value for that hour.The ANN forecast from the hours of 9 to 23, the forecasted outputs of ANN were very low that that of the actual values.Only on the hour of 24 that the forecasted data of the ANN has almost predicted the exact value of that hour.But as overall result, the forecasted outputs from the ANN model were very far from the actual values.The properties of the nonlinear estimators depend on the assumption that residual errors were independent and normally distributed with mean zero and correctly defined variance.Violations of this assumption can cause bias in parameter estimates, invalidate the likelihood ratio test and preclude simulation of real-life like data.The choice of error modelling is mostly done on a case-by-case basis from a limited set of commonly used models [7], [11], [16].This is basically why ANN residuals forecast has a very far-off prediction as to the actual residuals.www.ijacsa.thesai.orgThe comparison between the actual values and the ARIMA+ANN hybrid forecasted values is shown in Fig. 7. On the hours of 3, 6, 7 and 16, the forecasted outputs of the hybrid model was higher than that of the actual values.On hour 16, the forecasted value was much higher compared to all the models having a value 2000 more than the actual value.While on the hours of 2, 10 to 13 and 20, the forecasted values of the ARIMA+ANN hybrid model was lower than that of the actual values.Moreover, the rest of the hours almost had the exact value as the actual values with difference ranging from an estimate of 250 to 10.The ARIMA+ANN hybrid model outputs have the most values closest to the actual values of the three models.This was supported by study which also used ARIMA+ANN in which the hybrid model is able to perform well in terms of accuracy for every component model used in isolation [12].As for the overall comparison of results, the forecasted outputs of the ARIMA+ANN hybrid model was closer than that of the ARIMA and ANN models.The forecast difference of the ANN model can be the result of the data input since only one column was fed to the ANN model and because of its usage as an error model.The ANN model being used acts like an error model since its inputs are the residuals of the ARIMA model, but the error model itself can still be used as an individual forecasting model [12].Additionally, the researcher also stated in his study that the ANN error model may not give out good results if used individually.A separate study supports this assumption that the ARIMA+ANN hybrid model has better accuracy than the individual models because of how the ARIMA model caters to the linearity of the dataset and how the ANN model caters to the non-linearity of the same dataset [22].

IV. CONCLUSION AND RECOMMENDATIONS
This study attempted to implement and evaluate the performance of ARIMA, ANN and ARIMA+ANN hybrid models in predicting day-ahead electric load.A Java-based system was created which calls R for the ARIMA model and integrates Engoc library for the ANN model.Compared to the ARIMA(8,1,2) and the ANN model which used Resilient Propagation as the training algorithm and Hyperbolic Tangent as the activation function, the ARIMA+ANN hybrid model yielded the best forecasting performance with a MAPE value of 4.09% and a RMSE value of 1959.41ARIMA+ANN hybrid model also obtained an error rate which is below the acceptable tolerance error of 5%.Since the results of the ARIMA+ANN hybrid model has a lower MAPE than that of the ARIMA model and the ANN model, the hybrid model thus generate better result in prediction than solely using ARIMA and ANN.
This study only focuses on the ARIMA and ANN hybrid models; however, there are still other forecasting models that could also be viable in predicting day-ahead electric load.ARCH is one of the later model created that has the ability to read both linearity and non-linearity of data, however it is more into the linear side and could possibly be used in a hybrid model with ARIMA.Other hybrid models may also yield an even lower percentage error than that of this study and could be crucial in determining the better predicting model for an electric load dataset.R Statistical Studio and Encog library are two of the open source IDE and library, respectively, used in creating the hybrid model.R has no libraries that can allow it www.ijacsa.thesai.org to be integrated in Java and is solely able to predict linear data while Encog has limited training algorithm and activation function.The possibility that using other development environments or libraries can lead to a better forecast or using a system that can handle both linear and nonlinear predicting models might gain a better result than using separate ones.
The results of this research clearly suggest that the use of a hybrid model that caters the linearity and non-linearity of a dataset proves to be a better technique for a day-ahead electric load forecasting rather than the use of an individual model.

Fig. 1 .
Fig. 1.Diagram of the ARIMA and ANN implementation.

Fig. 5 .
Fig. 5. Comparison of the actual load data and the ARIMA forecast.It can be observed that in the hours between 1 and 10, the ARIMA model has higher forecasted values than the actual consumed electric load values.Meanwhile, in the hours between 10 and 15, the ARIMA model has lower forecasted values than the actual consumed electric load values.The values from 15 to 24 are very close to the actual values.Overall, the forecasted outputs from the ARIMA model were close to the actual values.

Fig. 6 .
Fig. 6.Comparison of the Actual Load Data and the ANN Forecast.

Fig. 7 .
Fig. 7. Comparison of the Actual Load Data and the ARIMA+ANN Forecast.A plot diagram shown in Fig. 8 shows the difference between all three models as to how each model are close to the actual consumed electric load values.From hours between 1 and 8, most of the forecasted values are higher than the actual values.ARIMA and the hybrid model are closest to the actual value while the ANN was farther away, except for hour 2. On hours of 9 to 22, most of the forecasted outputs were lower than the actual values, with the exception of hour 16 of the hybrid model and hour 17 of the ARIMA model which has values above the actual values.On hours 23 and 24, the values were near to the actual values with hour 24 of the three models almost captured the exact value.

Fig. 8 .
Fig. 8.Comparison of the Actual Load Data and Forecast of All Three Models.

TABLE III .
ADDING THE LINEAR AND NONLINEAR FORECASTS

TABLE IV .
MSE OF THE OUTPUT OF EACH MODEL On all of the hours, the 24th hour gained the lowest MSE than the other hours while the 13th hour gained the highest MSE.For the ARIMA model output, the 16th hour gained the smallest MSE while the 7th hour has the highest MSE.For the ANN model output, the 24th hour gained the lowest MSE, while the 13th hour has the highest MSE.Lastly for the hybrid model output, the 24th hour gained the lowest MSE while the 11th hour has the highest MSE.

TABLE V .
MAPE OF THE OUTPUT OF EACH MODEL

TABLE VI .
FORECASTED RESULTS FROM OCTOBER 21 TO 25 OF 2014