Evaluation Optimal Prediction Performance of MLMs on High-volatile Financial Market Data

The present study evaluates the prediction performance of the multi-machine learning models (MLMs) on high-volatile financial markets data sets since 2007 to 2020. The linear and nonlinear empirical data sets are comprised on stock price returns of Karachi stock exchange (KSE) 100-Index of Pakistan and currencies exchange rates of Pakistani Rupees (PKR) against five major currencies (USD, Euro, GBP, CHF & JPY). In the present study, the support vector regression (SVR), random forest (RF), and machine learning-linear regression model (ML-LRM) are under-evaluated for comparative prediction performance. Moreover, the findings demonstrated that the SVR comparatively gives optimal prediction performance on group1. Similarly, the RF relatively gives the best prediction performance on group2. The findings of study concludes that the algorithm of RF is most appropriate for nonlinear approximation/evaluation and the algorithm of SVR is most useful for high-frequency time-series data estimation. The present study is contributed by exploring comparative enthusiastic/optimistic machine learning model on multi-nature data sets. This empirical study would be helpful for finance and machine-learning pupils, data analysts and researchers, especially for those who are deploying machine-learning approaches for financial analysis. Keywords—Support vector regression; random forest; machine learning-linear regression model; optimal prediction performance; currencies exchange rates; stock price returns


I. INTRODUCTION
Since several decades, a lot of researchers and data analysts have been applying traditional econometric models (TEM) for hypothetical testing and financial-nonfinancial market evaluation. But there have some constraints with TEM as nonlinear approximation, future prediction developments of the markets and data prediction accuracy. On the other hand, nowadays, the supervised and unsupervised machine learning approaches have quite famous to precisely evaluate the different nature of big data. So, the analysis effectiveness is directly associated with the prediction accuracy the model.
The machine learning approaches are successfully employed into different domains with respect to their applications. But it is usually used for speech recognition, image processing, wind-speed frequency scaling, network filtering and financial markets prediction [1][2][3][4]. Furthermore, the evaluation of stock market and forex market returns with machine learning approaches are very hot topic nowadays.
Moreover, the financial-nonfinancial future markets predictions are very helpful for those who are willing for financing in the market and for hedgers to hedge their financial assets [5,6]. The machine learning algorithms (MLAs) have capability to analyze the linear-nonlinear data [7]. For example, the forex market is very dynamical market due to globalization. In prior era, the researchers and data analysts were used TEM to examine the forex market returns. But since last decade, the researchers and data experts have been rapidly diversifying from TEM to machine learning approaches for financial market prediction [8][9][10]. In fact, the MLMs are most powerful and very effective approaches to predict the high-volatile and nonlinear data financial data [11,12]. So, the key purpose of present study is to evaluate the prediction performance of underlying machine learning models on different nature financial data sets, such as the group1 comprised on stock price returns of KSE 100-index of Pakistan which is illustrated in Fig. 1. Fig. 1 indicates that the stock prince returns of KSE 100index is very dynamic financial market. This diagram is comprised on original data, rolling mean and average which collectively demonstrates the good returns (positive returns), bad returns (negative returns) and no returns (zero returns). The group2 contains on the PKR exchange rates against five major currencies (USD, Euro, GBP, CNY & JPY) which is illustrated in Fg. 2.   2 demonstrated that the PKR exchange rates against entire currencies have been steadily increasing over the period, which indicating the domestic currency (PKR) has been depreciating since 2007 to till 2020. Furthermore, these both empirical data sets will be evaluated by SVR, RF and ML-LRM. These empirical machine learning models are commonly used for classification and regression problems. But the present study will be executed the prediction performance of entire machine learning models on nonlinear and high-frequency data sets which quite effective for researchers and data analysts.
The remaining sections of study comprised as: the Section 2 will be contained on most similar literature on the stock markets and the forex markets which executed by MLMs. The Section 3 will be contained on data collection process, data preparation and mathematical interpretation and application of the models. Similarly, the Section 4 will be contained on results and discussion and forecast optimal prediction model. Finally, the Section 5 will be contained on conclusion-based sound evidence, limitation of the study and significant future developments of the study.
II. SIMILAR STUDIES Since the last decade, the researchers and data analysts have been rapidly diversified from TEM to flexible and experimental machine learning approaches for optimal prediction. In the modern era, the MLAs are typically used to determine the returns of dynamical stock markets. To predict the next day price index of the Taiwan index feature, [13] have deployed hybrid SVR with SOFT (self-organized feature map) approach by filter-based feature selection to improve the prediction accuracy of the model and mitigate the cost of training time. The findings of this study demonstrates that the hybrid SVR improves the training time and prediction accuracy. Another study has employed SVR for prediction stock prices on daily and up to the minute. The findings of this study shows that the SVR has absolute predictive power, whenever applicate the new strategy of model periodically [14]. The hybrid prophet-SVR model is outperformed than relatively other approaches by using time series demand with seasonality in stock index [15]. [16] have applied the machine learning based RF to examine the future predictions of stock prices of key listed firms. Moreover, [17] have been executed the trend of the Thai-stock market by deploying the SVR, multi-layer perception (MLP), and partial least square classifier models (PLSCM). The scholars have concluded that the Thai-stock market is rapidly growing while the investors are absolutely acquaintance about market dynamics. Similarly, some studies have evaluated the factors of stock prices returns via the reinforcement learning approach [18]. A lot of other studies have executed the real impact of prices by using mixed methodologies. Moreover, the Long-Short Term Memory (LSTM) is an artificial recurrent neural network (RNN) architecture which works on feedforward standards. Especially, it employed for outliers' detection problem in given big data set by fixing threshold, hence, this approach is very useful to investigate the dynamical stock market [19][20][21]. But the present study will execute the stock price returns of KSE 100-index of Pakistan and subsequent evaluate the optimal prediction performance model. Furthermore, the forex market is quite significant for those who want to be financing into the international market for 240 | P a g e www.ijacsa.thesai.org economic privileges. Because the currency exchange rates may bring probabilistic effects on economic value of their imperative assets. For currency risk management, a researcher community has been predicted the future currency exchange rates on voluntarily currency selection by using MLMs. For instance, The stochastic volatility model with jumps to SVR in order to account for sudden big changes in exchange rate volatility [22]. So the experimental studies demonstrate that the empirical new model has the ability to improve the forecast accuracy. The author in [23] had predicted the exchange rates of major currencies of developed counties. This study deployed the mixed methodologies as the deep neural networks, MLMs and econometric approaches to predict the currencies exchange rates. Besides, this study concludes that the currencies exchange rates of developing countries are more volatilized than developed countries. So, the traditional models may not properly investigate to the nonlinear approximation. According to econometric assumption, the empirical data should be in linear format for econometric modelling. But the empirical data lost their originality whenever it transformed from nonlinear to linear scale through the natural logarithm or another approach. But the MLMs have ability to evaluate every kind of data. Therefore, the usage of MLAs have been precipitously increasing over the period for optimal prediction to nonlinear approximation. The author in [24] has analyzed the couple of currencies exchange rates (USD/CHF, USD/CAD and USD/JPY) along with the shuffled frog leaping algorithm (SFLA). Moreover, a lot of other scholars executed MLMs prediction performance along with different currencies exchange rates data. Another study executed the currency exchange rate is a certain cause of domestic inflation by using the Dorn-busch approach. Besides, they also investigated that the USA housing prices raised 7% with excluded currency ERs. But in the last decade, the USA housing prices 40% grown-up by variation into local currency exchange rates [25]. After intensely viewed to the comprehensive literature, we summarized that the MLMs are excessively used to predict the stock market and forex market returns.

A. Data Collection and Preparation for Analysis
The empirical data set of the present study is divided into two certain groups. The group1 is contained on the stock price returns of KSE 100-index of Pakistan. In addition, the group1 data is dragged from the official website of the KSE of Pakistan (https://www.psx.com.pk/). This study measures the price returns from given stock prices as follow: Lumley [26].
In Equation 1, the stock price returns are executed as the 0 is daily stock prices at the time 0 subtract from the stock price 1 at time 1 and divided by 0 and multiplied by 100, whereas the 1 denotes to current stock prices and 0 denotes to the previous stock price of the stock market. As Fig. 1 is showing the stock price returns are high-volatile while this study follows [27][28][29] for normalizing the empirical data set.
Moreover, the group2 is contained on PKR exchange rate against five major currencies (USD, EUR, GBP, CHF, JPY). The thirteen-year daily currencies exchange rates data set dragged by the official website of the central bank of Pakistan (http://www.sbp.org.pk/). The Fig. 2 discretely exhibits the volatility of currency exchange rates against each given currency.

B. Research Methodology
The certain methodologies play an important role in the empirical studies to execute the hypothetical examination. But before to mathematical interpretation of the models, we must take keenly overview the whole process of performance evaluation of the models. Fig. 3 showed that the primarily both kinds of raw data sets are uploaded into the data repository of jupyter notebook and subsequently refine the data and scaled the data and become in specific format to optimize the machine learning algorithm. Furthermore, the scaled data set would split into train and test, whenever 80% will use for training purpose and rest of data will use for testing purpose. Furthermore, we shall train the model and evaluate the prediction performance of training model with average loss function. This study would execute the prediction performance of the models, if the train predicted error < expected error otherwise update the training data set.

1) Support vector regression:
The support vector regression model is popular as supervised learning approach which used for sort regression analysis [30,31]. The support vector machine (SVM) was primarily built for structural risk measurement, but subsequent the researchers and data analysts had customized it for regression purposes, especially for timeseries data. The SVM has the ability to measure linear and nonlinear approximation. The authors in [32] have employed the support vector machine (SVM) for the statistical problem. For the execution of the model prediction performance, the present study follows [33,34].
In Fig. 4, ∈ ℝ is an input function whereas − ℎ is − ℎ number of as = 1, 2, 3, ⋯ , . and = ∈ ℝ is an output function where − ℎ is − ℎ number of as = 1, 2, 3, ⋯ , . where − ℎ is vector at − ℎnumber of vectors which associated with specific weights � at − ℎ number of − ℎ vectors and summation bias. For instance, we have a time-series data set where = ( ), 1 ≤ ≤ . The core idea dragged by [35]. Let suppose = , the problem may solve as follows: In equation 3, the represents to specific weight at − ℎ number of vector and b denoted to biases. Moreover, is considered as input vector taken by φ into a higher dimensional space. The model performance can be improved by optimization the weights and bias of the model as below: In equation 4, the C is makes swapping between two models' simplicity and generalization ability. If − � ( )� − ≤ ξ + 1 and � ( )� + − ≤ ξ + 1 , the error position is determined by slack variables as ξ & ε i . We can map high volatile and nonlinear data set from the original vector space by using the kernel approach. However, this is a pretty way to drag the SVR model as below: Whereas.
Here α 1 α 2 both are represented to langrage variables multiplier. The below equation indicates that the extensive utilization of kernel by Gaussian Function concerning the size of σ 2 : 2) Random forest: The RF is an ensemble approach which keeps the capability of performing as both regression and classification problems. In addition, it can handle binary and multi-classification problems. Moreover, the present study uses bootstrap aggregation for absolute random selection as follows [36]. For instance, when the explained variable is a continuous variable and is getting random vector then it promotes to decision trees of RF for regression purpose. Moreover, the x and y are random vectors and training data set taken systematically. Besides, h i (x) is a single decision tree from multi decision trees where − ℎ denote the number of decision trees for prediction ℎ( , ) , where the − ℎ number of outputs by − ℎ random vectors as = 1, 2, 3, ⋯ , as follows [37].
In equation 7, the ( ) demonstrates the output of combined RF regression models. Moreover, in the training data set − ℎ shows the number of values represented in the model.
For example, A is the outcome from classifier ℎ ( from random vectors that can be optimum ensemble as p̂(A) = proportion of classifiers ℎ (1 ≤ ≤ , whenever event A occurs. In addition, the analysts acknowledge that they optimize the model prediction performance by making more generalizations to stochastic error (e). * = , (∫( , ) < 0 According to the theorem, the stochastic problem can be determined as below equation whenever the number of decision trees may increase. * − ∞ �⎯⎯� , � ⊝ (ℎ( ,⊝) = ) ≠ ⊝ (ℎ( ,⊝) = ) < 0� (11) In equation 11, the , denotes problem in , random vectors and − ℎ denotes the total number of decision trees in the model. In addition, if the decision trees grow up the generalization of PE will tend to upper bond. Hence, the RF algorithm has worthy convergence and can prevent overfitting [38,39]. 242 | P a g e www.ijacsa.thesai.org

3) Machine learning-linear regression model:
The regression approach is very useful for linearity approximation. Although the traditional approaches have been employing for execution the high-frequency and nonlinear data sets, but traditional approaches (linear regression) cannot properly investigate the nonlinear data sets [12]. However, the present study is applicating the ML-LRM Fig. 5 for best prediction performance. For instance, is input parameter where − ℎ is − ℎnumber of input parameters as i = 1, ⋯ ,2, ⋯ ,3, ⋯ and every input parameter takes specific weight which denotes by with = 1, 2, 3, ⋯ , at − ℎ number of weights. At the time of training model, the weight multiplied with input parameters and summation biases weights. Moreover, the nth denotes the number of values of the training model. Primarily, we train our model on training data set and test the model output by using the testing data set and execute the prediction performance of the model.
In mapping function Ф makes typical function with x input parameter as Φ: X → ℝ N . A linear function of hypothetical set: weight and bias are associated with mapping function as → . (x) + b: w ∈ R N , b ∈ ℝ. Moreover, the exclusive weight leads to risk minimization and optimization of the typical model. So, Additionally, the convex and differentiable functions executed as: Equation 17 is explained that if the w = 0, then the hypothesis set would also be zero / no fraction between inputoutput parameters of the model. Table I summarizes the significant information of both group data. The group1 is contained on daily stock price returns of KSE 100-index of Pakistan and group2 is contained on PKR currency exchange rate against five major currencies of the world. The above statistics demonstrated that the total number of values are 3187 and 3464 in group1 and group2, respectively. Moreover, in group1, the mean and standard deviations are 0.04 and 1.19, respectively, indicating that the group1 data set is highly deviated over the period. In group2, the mean of currencies exchange rates is 97.06, 121.7, 146.9, 98.4 and 0.96, and the standard deviation of currencies exchange rates is 23.4, 21.3, 20.9, 26.9 and 0.22, respectively. According to group 2 statistics, the PKR has been depreciated since 2007 to 2020. Furthermore, the other descriptive statistics are illustrated in the above table as like minimum, maximum, etc.

IV. RESULTS AND DISCUSSION
Table II executes the augmented dickey fuller (ADF) approach to investigate whether the particular parameters have unit-root (non-stationary) or not (stationary) at specific critical level. Although the machine learning algorithms have capability to analyze the nonlinear approximation, but with respect to econometric assumptions, the empirical data should be to stationary (normal distributed) before to diagnosed. So in Table II, the statistics demonstrated that the t-value > critical value at 1%, 5% & 10% confidence intervel at 1 st difference. However, the unit root is not existed in the significant model.    Tables III, IV and V execute the prediction performance of MLMs on different nature of data sets. In 1st experiment, the present study used daily stock price returns of KSE 100-index to evaluate the prediction performances of MLMs through mean absolute error (MAE), mean Squared error (MSE) and root mean squared error (RMSE). Consequently, the output shows that the SVR gives relatively optimal prediction performance than RF and ML-LRM on group1 data set; hence, the MAE, MSE and RMSE of SVR is less than RF and ML-LRM. But the ML-LRM is giving best prediction than RF on the corresponding data group. So, on the 1 st experiment, the absolute prediction performance is summarized as SVR > ML-LRM > RF. Furthermore, in 2nd experiment, we used group2 data regarding currencies exchange rates for the prediction performance of MLMs. In addition, the loss of RF < SVR < ML-LRM while the RF is giving optimal prediction performance than SVR and ML-LRM. In addition, the SVR gives best prediction than ML-LRM at the same experiment. In 2nd experiment, the prediction performance of MLMs is categorically assembled as RF > SVR > ML-RM. In fact, the empirical data set is nonlinear nature in 2nd experiment and linear regression algorithm is operating in ML-LRM while relatively the ML-LRM doesn't properly investigate the data. On the other hand, the RF is typically used for classification and regression problems. So the RF algorithm is best recognizing the typical data set, even though the data is nonlinear. However, the RF is comparatively given the optimal prediction performance in 2nd experiment. Fig . 6 executes the graphical prediction performance of SVR, RF and ML-LRM at group1. The output demonstrated the SVR model is given optimal prediction than RF and ML-LRM, hence, the red line is very closed to true line. But the RF predicted plots are slightly away from true line. Similarly, the ML-LRM prediction line is comparatively less close to true line. So, the whole prediction performance of MLMs from Fig. 6 executes as follows: SVR > ML-RM > RF. Thus, the SVR can comparatively best predict than RF and ML-LRM on a high-volatile time series data set. Some previous studies are supporting to findings of current study as: [40] determined the comparative prediction performance between SVR and ensemble empirical mode decomposition (EEMD) through Singular Spectrum Analysis (SSA) on Shanghai stock price index. Moreover, this study correspondingly examining the market tendency, market fluidity, economic topographies of the market. The output shows that the SVR can relatively best predict to the fluidity of forex market and stock market than EEMD. So, this is sound evidence about the SVR can gives more efficient and accurate prediction than generic TEM and MLMs on high-frequency data which supported to findings of current study.
The Fig. 7 executes the graphical prediction performances of five empirical models on group2. Each model has relatively summarized the prediction performance of SVR, RF and ML-LRM. In addition, the output of every model is categorically presented along with specific color as follows: SVR with dark-red, RF with yellow-green, ML-LRM with dark-orange and original input data with light-blue color. The model 7(a) in our 2 nd experiment exhibits the prediction performance of MLMs by taking the PKR exchange rate against USD on the y-axis and the number of time steps on the x-axis. Consequently, the RF is giving relatively best prediction than SVR and ML-LRM while yellow green line is extremely closed to the true line. In model 7(b), again the RF is relatively giving the best prediction while the yellow green line is very closed to light blue line. Similarly, the output of models c, d & e is accompanied as previous two models. With respect to our 244 | P a g e www.ijacsa.thesai.org 2 nd experiment, the findings concluded the RF can comparatively best predict to the nonlinear approximation than SVR and ML-LRM. The findings of 2 nd experiment is supported by [41].

V. CONCLUSION
The stock market and forex market both are highly dynamical financial markets. The usage of machine learning approaches are rapidly growing in financial markets for economical analysis. Therefore, the key purpose of the present study is to investigate the best predictor machine learning model by giving different nature of high frequency financial data sets. So, the empirical findings demonstrated that the SVR and RF have ability to optimistic prediction on highvolatile and nonlinear data sets respectively. In fact, the SVR is a supervised learning approach which works on the principle of support vector machine (SVM) to solve the classification and regression problems. Hence, the SVR is very useful approach for complex and high volatile data. On the other hand, the RF is usually used for classification problems. However, the RF has given best prediction on nonlinear approximation. Moreover, this study had targeted only forex market and stock market of a country, hence, the study may more robuster by using panel financial market data. Further research directions are using hybrid deep neural networks with filter-based feature selection to improve the prediction accuracy and reduce the cost of training time and compare the performance with RF and SVR by using more complex financial data.