Using Multiple Seasonal Holt-Winters Exponential Smoothing to Predict Cloud Resource Provisioning

Elasticity is one of the key features of cloud computing that attracts many SaaS providers to minimize their services' cost. Cost is minimized by automatically provision and release computational resources depend on actual computational needs. However, delay of starting up new virtual resources can cause Service Level Agreement violation. Consequently, predicting cloud resources provisioning gains a lot of attention to scale computational resources in advance. However, most of current approaches do not consider multi-seasonality in cloud workloads. This paper proposes cloud resource provisioning prediction algorithm based on Holt-Winters exponential smoothing method. The proposed algorithm extends Holt-Winters exponential smoothing method to model cloud workload with multi-seasonal cycles. Prediction accuracy of the proposed algorithm has been improved by employing Artificial Bee Colony algorithm to optimize its parameters. Performance of the proposed algorithm has been evaluated and compared with double and triple exponential smoothing methods. Our results have shown that the proposed algorithm outperforms other methods.


INTRODUCTION
Elasticity feature plays an important role in cloud computing by allowing SaaS providers to allocate and deallocate resources to their running services according to the demand.Elasticity allows SaaS providers to pay only for resources that are used by their cloud services [1].However, the delay between requesting new resources and it being ready for use violates Service Level Agreement [2].Therefore, forecasting future resource provisioning is needed to request resources in advance.
Exponential Smoothing is a very popular smoothing method and has been used through years in many forecasting situations [3].Many researchers have exploited Exponential smoothing methods to predict future resource provisioning for cloud computing applications [4] [5].However, most of them have used double exponential smoothing, which cannot model workloads if there are seasonalities.
Most of cloud-computing applications' workloads are influenced by seasonal factors (e.g., day, week, month, year) and have more than one seasonal pattern [6][7] [8].Workload has intraday seasonal pattern if there is a similarity of request when comparing requests of the corresponding hour from one day to the next day.Intraweek seasonal pattern exists if there is a similarity between requests in two corresponding days from two adjacent weeks [3].Therefore, there is a strong demand to use predictive approach that is able to capture all seasonality patterns.This paper proposes resource usage prediction algorithm, which extends Holt-Winters exponential smoothing (HW) method to model multiple seasonal cycles.However, modeling multiple seasonal cycles requires large number of observation values.For example, predicting resource usage with intraday, intra-month, and intra-year seasonality patterns requires at least two years observation values.Moreover, finding optimal parameter values (smoothing constant, trendsmoothing constant and seasonal-smoothing constants) for multiple seasonality model is not an easy task.
Therefore, the proposed algorithm detects seasonality patterns from available historical data by applying seasonality test, and extends HW accordingly to model detected seasonality patterns.While historical data size grows up and more seasonality patterns are detected, HW is gradually extended to be able to model detected seasonality patterns.Furthermore, prediction accuracy of the proposed algorithm has been enhanced by using artificial bee colony algorithm to find near optimal values for its parameters.Thus, unlike most of current resource prediction approaches, the proposed algorithm does not require any minimum number of observations values before applying it.However, good prediction accuracy will not be achieved until several steps have been made.
The proposed algorithm has been evaluated using CloudSim simulator with real Web server log called Saskatchewan Log [6].Performance of the proposed algorithm has been compared with double and triple exponential smoothing methods.Experimental results have shown that the proposed algorithm outperforms algorithms that use double or triple exponential smoothing methods.This paper is organized as follows.In Section II related works are overviewed.The proposed algorithm is presented in www.ijacsa.thesai.orgSection III.Performance of the proposed algorithm is evaluated in Section IV.Finally, Section V concludes.

II. RELATED WORK
The problem of predicting resource provisioning in cloud computing has been studied extensively over the last few years.Several prediction techniques have been used to predict cloud resource provisioning.However, most of current approaches do not consider multi-seasonality in cloud workload, and most of them use prediction techniques that do not have ability to model more than one seasonal cycle [4][9][10] [11].
Islam et al. [12] have proposed framework to predict future resource usage in the cloud.The proposed framework uses two machine-learning algorithms (Neural Network and Linear Regression) with sliding window and cross validation techniques to predict cloud resource usage.The proposed framework is evaluated by using dataset that is collected by using TPC-W benchmark.Statistical metrics is proposed to assess prediction accuracy.However, the proposed framework uses three layers feed-forward Neural Network, which does not able to predict resource utilization when there are long time lags between events.Moreover, the proposed framework is tested with data that are collected from 135 minutes, which does not contain any seasonality.Therefore, prediction with seasonality is not examined.Kanagala and Sekaran [4] have proposed dynamic threshold-based auto-scaling approach that considers virtual resource start-up and stabilization delays.Virtual resource utilization is predicted by using double exponential smoothing method, thresholds are adapted based on the predicted resource utilization to minimize violation of Service Level Agreement.However, double exponential smoothing method cannot be used to model seasonality.
In [5], Huang et al. have proposed resource utilization prediction model based on double exponential smoothing method.Prediction accuracy of the proposed model has been evaluated using CloudSim simulator, which shows that double exponential smoothing has better prediction accuracy than simple mean based method and weighted moving average method.However, smoothing constant and trend-smoothing constant are determined using trial method, which does not grant quality of the final solution.
Although, seasonal linear regression can be used to predict workload with seasonality, most of current approaches do not consider cloud workload seasonalities and use conventional linear regression to predict cloud resource utilization [1][13][14] [15].In [16], Yang et al. have proposed cost-aware auto-scaling approach, which predicts workload using linear regression model.The problem has been formulated as integer programming problem and solved using greedy heuristic to reduce costs.The proposed approach uses vertical and horizontal scaling methods.Allocated resources are scaled vertically by creating virtual machines on the same cluster node or using unallocated resources available at a particular cluster node to scale up a VM executing on it.Horizontal scaling is used to create virtual machines on other cluster nodes.
To gain benefits from several time series prediction models, Messias et al. [2] have proposed cloud workload prediction methodology that combines several time series forecasting models using genetic algorithm.Each time series prediction model has been assigned a weight, and genetic algorithm adapts the assigned weights to find the best weight combination that maximizes prediction accuracy.
Wei and Blake [17] have proposed an algorithm to predict future resource requirement in the cloud.The proposed algorithm uses five prediction models and differentiates between these models using root-mean-square-error (RMSE).Prediction model with the lowest RMSE is used to predict future resource requirement.Although, the proposed algorithm uses prediction techniques that do not have ability to model seasonality, it can be extended to include more prediction techniques with the ability to model seasonality.
Salah et al. [18] have proposed analytical model based on Markov chains to predict minimal number of VMs and load balancers required to satisfy Service Level Agreement such as throughput and response time.The proposed model has been validated using experimental testsbed deployed on the Amazon Web Services.Discrete-event simulation has been used to verify correctness of the proposed model.

III. PROPOSED ALGORITHM
Although many researchers have employed double exponential smoothing for forecasting cloud applications' workload [5][4], double exponential smoothing does not able to model seasonality [3].HW can be used for forecasting seasonal workloads [3].However, HW is only able to model workloads with one seasonal pattern and cloud applications' workloads may have more than one seasonal pattern (e.g., intraday, intraweek, intra-month, intra-quarter, intra-year).
Therefore, in this paper, HW has been extended to be able to accommodate multi-seasonal patterns.As shown in equations 1-5, HW has been extended by adding seasonal indices and smoothing equation for each seasonal pattern.

{ ∏ ̂
where is an index denoting a time period, is smoothed value at time , is observed value at time , is trend factor at time , is seasonal indices for seasonality pattern , is number of periods in a completed seasonal cycle for seasonality pattern α is the smoothing constant, the trend-smoothing constant, is seasonal-smoothing constant for seasonality pattern s number of seasonality patterns, and ̂ is the k-step-ahead forecast at time .www.ijacsa.thesai.orgInitial smoothed value for the level, , is calculated as average of the first periods, which are periods in the first two cycles from the first seasonal pattern.If there is no seasonality patterns, is initialized by the first observation value .Initial value for the trend factor, is calculated as of average of the difference between first observations and second observations.If there is no seasonality patterns, is initialized as a difference between second observation value and first observation value .
For each seasonal cycle, at least three completed seasonal data are required to initialize its seasonal indices.Seasonal indices are initialized as average of ratios of observed value to its centered moving average (calculated from periods around observed value), taken from the corresponding period in each of the first two completed seasonal data, which starts from .For example, seasonal indices for seasonality pattern are calculated as following: where is centered moving average around for periods

∑ ⁄
Finally, artificial Bee colony algorithm is applied to determine near optimal values for smoothing constant, trendsmoothing constant and seasonal-smoothing constants that minimize Mean Squared Error (MSE).

∑ ̂
Algorithm 1 shows steps of the proposed algorithm.The first input is initial list of observation values that contains 60 observation values (from 60 minutes).This number of observation values is specified to start with enhanced accuracy.The second input is the list of completed seasonal cycles' length of expected seasonal patterns.Instead of applying seasonality test periodically, the second input specifies time points to test existence of seasonality patterns.The outputs are list of predicted values and list of seasonal cycles' length of detected seasonal patterns.
In the first line, initial smoothed value is set to the observed value , and initial trend factor is set to .Best values for smoothing constant α and trendsmoothing constant are obtained by using Bee Colony Algorithm (Algorithm 2).At this point, number of seasonal cycles .Therefore, equations 1-5 are minimized to the following equations, which represent equations associated with Double Exponential Smoothing.

̂
Therefore, prediction accuracy of the proposed algorithm during the interval from (where is the number of periods in completed seasonal cycle for the first seasonal pattern) is very similar to prediction accuracy of double exponential smoothing.

If equals to
, where , seasonality test is applied to check if the list of observed values has seasonal pattern with length or not.New seasonal pattern is detected if autocorrelation coefficient is greater than or equal 0.3.Length of the detected seasonal pattern is added to the list of seasonal cycles' length , and number of detected seasonal patterns is increased.List of seasonal indices for the new seasonal pattern is calculated and added to .Smoothing constant, trend-smoothing constant, and seasonal-smoothing constants are updated to the near optimal values using artificial Bee colony algorithm.Finally, extended formula is employed to predict future required resources.
Algorithm 2 shows steps of determining best values for smoothing constant α, trend-smoothing constant , and At the beginning, initial population is initialized with scout bees, which are randomly scattered across solution space.Here, all constants (smoothing constant α, trendsmoothing constant , and seasonal-smoothing constants ) are greater than zero and less than or equal one.For each scout bee in , flower patch is delimited that contains its neighborhood.
MSE is calculated for each scout bee by applying equations 1-5 from to ‖ ‖, where is number of periods in completed seasonal cycle for the largest seasonal pattern.if , MSE is calculated by applying equations 1-5 from to ‖ ‖.Scouts are sorted in ascending order according to their MSE.

Best sites
with lowest MSE are selected from , and elite sites with most lowest MSE are selected from .
Each scout in performs waggle dance to recruit forager bees to search further in its flower patch.Such that, number of recruited forager bees to ( ) is greater than number of recruited forager bees to the remaining best sites ( ).
To find fittest bee of each flower patch, recruited forager bees are randomly distributed in flower patch.MSE is calculated for each bee.If there is recruited forager bee with MSE lower than MSE of its scout bee, fittest bee will be selected as a new scout.Otherwise, flower patch will be shrunken around its scout.After pre-specified number of search cycles, the fittest bee of each flower patch is returned as a local optimal solution.New solutions are generated randomly for non-best sites , and all scout in are sorted in ascending order according to their MSE.This search cycle will be repeated until reaching termination condition.Finally, values of smoothing constant α, trend-smoothing constant , and seasonal-smoothing constants are obtained from fittest scout bee in current population.

IV. PERFORMANCE EVALUATION
To evaluate performance of the proposed algorithm, its performance have been compared with double and triple exponential smoothing methods.The following subsections, describe evaluation environment settings and discuss simulations' results.

A. Evaluation environment settings
The proposed algorithm has been evaluated using real Web server log called Saskatchewan Log [6].Saskatchewan log contains HTTP requests to the University of Saskatchewan's WWW server, which is located in Saskatoon, Saskatchewan, Canada.This log was collected from 00:00:00 June 1, 1995 to 23:59:59 December 31, 1995, a total of 214 days [6].
Cloudlets have been generated according to Saskatchewan log and sent to CloudSim simulator.For each minute, CloudSim simulator calculates total required CPU to process incoming requests without violating Service Level Agreement.The proposed algorithm receives required CPU as observed value and predicts required CPU after k-minutes.K has been set to 15, where k is a virtual machine startup delay.
To evaluate accuracy of the proposed algorithm, three evaluation metrics have been used: -Mean absolute percentage error (MAPE), which is defined as following: where is mean absolute percentage error at time t, ̂ is the k-step-ahead forecast at time , and is observed value at time .A smaller value of implies a better prediction accuracy.
-Percentage of predictions within 25% (PRED(25)), percentage of prediction within 25% at time is defined as following: -Root Mean Squared Error (RMSE), RMSE at time is defined as following: √ ∑ ̂ A smaller value of implies better prediction accuracy.

B. Evaluation results
Although, the proposed algorithm has been evaluated with many real workload traces such as [6][7] [8] in this evaluation only one of them has been shown, which is Saskatchewanhttp.
Fig. 1 compares the proposed multi-seasonal algorithm with double and triple exponential smoothing methods using Mean Absolute Percentage Error (MAPE), which has been defined in the previous section.As shown in Fig. 1, MAPE of the proposed multi-seasonal algorithm stays below 29% while triple and double are above 44% and 135% respectively.Fig. 2 shows that more than 57% of predicted values by using the proposed multi-seasonal algorithm are with prediction error less than 25%.In another side, 38% of triple exponential smoothing predictions are within 25%, and 8-18% of double exponential smoothing predictions are within 25%.Finally, Root Main Square Error of the proposed multi-seasonal algorithm has been compared with double and triple exponential smoothing methods in Fig. 3, which shows that RMSE of the proposed multi-seasonal algorithm is better than other methods.As future work, long short-term memory recurrent neural networks will be incorporated with the proposed algorithm to predict cloud resource utilization when there are very long and variant time lags between events.Because, in seasonality patterns, seasonal cycle length is considered constant for each seasonal pattern.However, in some cases, lags between events are variant and have to be considered during prediction.

Fig. 1 .Fig. 2 .
Fig. 1.Mean Absolute Percentage Error comparison Select best sites  from  9: Select elite sites  from  10: Recruit forager bees to  and   11: Apply local search to find fittest bee of each flower patch 12: Generate random solutions for non-best sites   13: Calculate MSE for non-best sites 14: Sort all scouts in in ascending order based on their MSE values 15:    of the first scout in the sorted  16: end while 17: Determine constants' value according to fittest scout in population  18: return End www.ijacsa.thesai.orgvalues are between 0 and 1. Prediction will be more effective if value is closer to 1.