A Hybrid Data Mining Approach for Intrusion Detection on Imbalanced NSL-KDD Dataset

Intrusion detection systems aim to detect malicious viruses from computer and network traffic, which is not possible using common firewall. Most intrusion detection systems are developed based on machine learning techniques. Since datasets which used in intrusion detection are imbalanced, in the previous methods, the accuracy of detecting two attack classes, R2L and U2R, is lower than that of the normal and other attack classes. In order to overcome this issue, this study employs a hybrid approach. This hybrid approach is a combination of synthetic minority oversampling technique (SMOTE) and cluster center and nearest neighbor (CANN). Important features are selected using leave one out method (LOO). Moreover, this study employs NSL KDD dataset. Results indicate that the proposed method improves the accuracy of detecting U2R and R2L attacks in comparison to the baseline paper by 94% and 50%, respectively. Keywords—intrusion detection system; feature selection; imbalanced dataset; SMOTE; NSL KDD


INTRODUCTION
With the advance of computer science and network technology, access to internet has become an important part of people's daily life.Moreover, the number of individuals connecting to the internet is increasingly growing and this makes network security a challenging issue.Previously, authentication, data encryption, and firewall techniques were used to protect the security of computers [1].
Today intrusion detection systems (IDSs) are used to protect computer systems from the risk of threats.Generally, IDSs divide into signature-based and anomaly based detection.In signature-based detection, packets are scanned to search for a set of events previously detected as attacks.However, in anomaly-based detection, intrusion detection systems exploit behavior patterns.In fact, a profile of normal behavior is created and any deviation from this behavior is considered an anomaly.Initial intrusion detection systems widely used signature-based intrusion detection methods.However, they had a high false alarm rate.Thus recent methods are based on behavior modeling and employ data mining methods, statistical analyses, and artificial intelligence techniques to detect anomalies [2,3].
At initial stages, most intrusion detection studies employed rule-based expert systems and statistical approaches [4].But with advent of larger datasets the results of rule-based expert systems and statistical methods became challenging issue.Therefore, several data mining techniques were introduced to overcome this problem [5,6].
Recently, a combination of techniques is exploited to improve detection rate and accuracy of IDSs [7,8].Shin, Lee, Kim, and Kim in [9] introduced a novel probabilistic approach to forecast and detect network intrusions.In this approach a Markov chain was used for probabilistic modeling of abnormal events in network systems.This is done by performing Kmean clustering, and introducing the concept of an outlier factor.Result showed that the proposed approach achieves high detection performance.
In recent years, due to the large data sets related to intrusion detection, feature selection methods (FS) are taken into consideration.FS is a process of choosing an optimal subset of features that represents the whole dataset [10].Canedo, Marono, and Betanzos [11] proposed a new combinational method of discretization, filtering and classification which is used as an FS to improve the classification task, and they applied this method on the KDD Cup 99 dataset.Lin, Ying, Lee, and Lee [12] presented an intelligent algorithm which was applied to anomaly intrusion detection.It used simulated annealing (SA) and support vector machine (SVM) to find the best feature subsets, while SA and DT were proposed to generate decision rules to detect new attacks.Eesa, Orman, and Brifcani [10] proposed a new feature-selection approach based on the cuttlefish optimization algorithm (CFA) in order to improve performance of intrusion detection systems (IDSs).In this paper CFA used as a feature selection tool and the decision tree (DT) classifier as an evaluator on the selected features that are produced by the CFA.The performance of the proposed approach is evaluated through well-known KDD'99 data set.The results show that the feature subset extracted based on CFA gives a higher detection rate and accuracy rate with a lower false alarm rate compared to the obtained results using all features.
Several studies are done in this scope but there is no exact answer which features subsets are optimal or more representative.Also, the time taken for training the systems and for the detection task to further validate their systems is not considered in many evaluation methods.Recent systems that combine or integrate multiple techniques require much greater computational effort.As a result, this can degrade the efficiency of 'on-line' detection [1].
Lin, Ke, and Tsai in [1], tried to eliminate these limitations by combining Kmean clustering and 1-NN search algorithm.Also, execution time is reduced considerable.This approach named CANN (cluster center and nearest neighbor) is a basis *Corresponding author: mr.parsaei@yahoo.comwww.ijacsa.thesai.org of this paper and will introduce later in detail.But a drawback of this approach is its poor capability in detecting U2R and R2L attacks.
In some studies, overall detection accuracy was measured and showed high DR rate, but the accuracy of each attack class was not provided separately [13][14][15].So, decide whether these methods are able to detect U2R and R2L attack classes are not possible.
The number of records in the R2L and U2R attack classes is very smaller than that of normal and other attack classes in datasets, which leads to an imbalanced problem.Each dataset with a significantly uneven data distribution between its classes can be considered an imbalanced set.Particularly, this imbalance is considered a between-class imbalance in which one class has much less instances than other classes.Classification of an imbalanced data set is a challenging issue for researchers.Most standard data mining techniques consider balanced data set and when they work with imbalanced data set, results are biased toward numerous majority class samples.So the accuracy of classification for majority class is high and is low for minority class [16].
Khor, Ting, and Amnuaisuk in [17] used under-sampling and oversampling methods to mitigate the rare class problem.First the dominant classes, i.e.Normal and DoS were undersampled to balance the class and then used SMOTE for oversampling decreased dataset.Then the improved data set was evaluated by major learning algorithms such as Naïve Bayes Classifier, Bayesian Networks, and Decision Trees algorithms such as ID3, C4.5 and Classification and Regression Trees (CART).But these two methods were less effective in mitigating the problem because there is no significant improvement on detecting R2l and U2R.
This study aims to investigate the effect of SMOTE coupled with CANN approach which proposed by Lin et al. [1] to improve detection rate of low frequency attacks like R2L and U2R.
The rest of this paper is organized as follows.Section II introduces SMOTE and CANN methods.Section III explains the dataset, LOO feature selection, and evaluation parameters.Section IV present the experiments.Finally, section V conclude the paper.

A. Synthetic Minority Oversampling Technique (SMOTE)
Chawla, Bowyer, Hall, and Kegelmeyer [18] proposed a novel method called SMOTE to overcome the imbalanced data problem.SMOTE increases the number of minority instances by creating new synthetic instances instead of repeating minority samples.In this method, new synthetic instances are created based on two parameters, oversampling rate (%) and the number of K nearest neighbors.Creating new synthetic samples are related to features type (continues or nominal).If the features of the instance are continuous, the distance is calculated between the feature vector of a sample in the minority class and one of the k nearest neighbors.The computed distance is then multiplied by random number δ in range zero and one.Finally, the obtained value is added to the value of the initial vector [19].The new feature vector is calculated as follow: Where, is the new synthetic instance, is the feature vector of each instance from the minority class, is the th selected nearest neighbor for and δ is a random number between 0 and 1.
Producing random instances for nominal features is as follows [20]: first the majority vote method is used among nominal features of the considered instance and its k nearest neighbors.If equal, one is selected randomly.In the next stage, the selected values are considered as the synthetic instance.

B. CANN (Cluster center and Nearest Neighbor)
Lin et al. [1] proposed a novel feature representation approach, namely the cluster center and nearest neighbor (CANN) approach.It works based on two distance.Since cluster centroids are a good representation of the entire data, the data is first clustered by using Kmean to extract the cluster centroids.
Next step is to measure and sum the distance (dis1) between all data of the given dataset and the cluster centers and the distance (dis2) between each data point and its nearest neighbor in the same cluster.This leads to a new distance based feature value to represent each data point of the given dataset.These two distances are added according to equation 2 and a new feature is achieved.
Where, is the ith records in the dataset, is the jth cluster centroid which is extracted by Kmean, and is the nearest neighbor of .Therefore, an n-dimensional records is converted to a single-dimensional one.After running the algorithm this new and one-dimensional distance based feature is used to represent each data sample for intrusion detection by a k-nearest neighbor (k-NN) classifier.

III. IMPLEMENTATION
This study aims to improve the ability of intrusion detection systems in detecting U2R and R2L attacks by proposing a hybrid approach which is a combination of synthetic minority oversampling technique (SMOTE) and cluster center and nearest neighbor (CANN).Moreover, this study utilizes the NSL-KDD dataset.In this study the dataset with 41 datasets is first reduced to a smaller dimensional set with 21 features using LOO method.

A. Dataset
Statistical analyses on KDD CUP 99, showed that this dataset has weaknesses that effect on systems` performance.Its major weakness is its repetitive records, which causes a bias towards frequent data.After investigating and analyzing this set, it was known that 78% of the training data and 75% of the test data are repetitive [21].Therefore, this study uses NSL www.ijacsa.thesai.orgKDD 1 .The total number of records in this dataset is 125973, where 67343 records are normal data and the rest indicate attacks.The total number of features is 41, which include numeric, nominal, and binary features.Table I presents the features, as well as their types and numbers.The training and test datasets include 21 and 37 different attacks, respectively.Some unknown attacks are placed in the test set which are not in the training set to evaluate the ability to detect these unknown attacks.

B. Feature Selection
Most fields in dataset are redundant.This number of dimensions increases the processing time and in some case degrade systems performance.This study by using leave-oneout (LOO) approach extracts important features from a total number of 41 features.The importance measure of each feature is evaluated based on the two parameters of accuracy and false The LOO algorithm is executed 41 times for the 41 features.Number of the resulted important features after executions is as follows: 1, 9, 10,11,12,13,14,15,17,18,21,22,25,29,30,31,32,36,39,40,41.

C. Evaluation Parameters
This study uses some assessment metrics such as accuracy, detection rate, and false alarm rate as evaluation parameters, which are computed based on the confusion matrix in table III.

IV. EXPERIMENTS AND RESULTS
Figure 1 presents the proposed algorithm.First by using LOO, important features are extracted from the training dataset.Afterward, the training set is sampled 10 times, each time, the dataset is balanced using SMOTE method, and then CANN is used to classify the dataset and build a model.For evaluations, the test set is applied to the model and finally, the average of 10 executions is computed.This study uses 20% of NSL KDD dataset with stratified sampling.More specifically, each class is sampled in proportion to the percentage it covers in the original dataset.For instance, if class DoS covers 36% of NSL KDD records, the same percentage of records is still covered by the DoS class after sampling.Of course, due to the www.ijacsa.thesai.orgsmall number of R2L and U2R records, first they are all placed in the samples and then the other three classes were sampled.Sampling was performed 10 times by changing the seed of the random number generator.Since samples are random, the accuracy of one execution of the algorithm on one set cannot be an indicator of its accuracy on the entire data.Therefore, sampling is repeated 10 times and 10-fold cross validation is used to evaluate each execution of the algorithm.
The dataset used in this study is not the same as the baseline research [1].In order to making the obtained results to be comparable with those of the baseline study, its experiments were implemented again using the considered dataset.Related codes were written using MATLAB and all data mining tasks were performed using WEKA 6 software.In [1], the experiment was conducted on two feature sets, once with 6 features and again with 19 features, where the latter improved the ability to detect U2R and R2L attacks.This study compared with proposed method with the CANN model with 19 features.
Since training records are randomly selected, the algorithm`s accuracy changes for different records as the training set.A solution is to use probabilistic boundaries.Equation 6shows that the probability that accuracy be in the confidence interval is 1-α.If random quantity { } has normal distribution with mean µ and standard deviation σ, the average sample obtained by random sampling with size n has a distribution with mean µ and standard deviation σ, which tends to a normal distribution by increasing n (equation 7).Therefore, equation 8 can be used to obtain the confidence interval for accuracy [22], where X is the mean accuracy of repeating the model by changing the seed of the random number generation and S is the standard deviation of accuracy for 10 executions.
nn ss X t X t nn Since the seed of the random number generator is changed at each iteration and numbers are independent, we can assume that they have a normal distribution.Therefore, we can say that each iteration is performed independently and the achieved numbers have a normal distribution.Considering the 10 executions performed by changing the seed of the random number generator, tables VI and VII present the results.All parameter values are averaged over 10 executions.
Since tests and measurements are performed under the same conditions, paired systems are used to compare the results of the two approaches.Accordingly, equation 8 can be used to compare the two methods [22].With confidence degree α of 99% and considering the number of samples as n=10,   0/ 995;9 t is 3.250.Confidence interval is obtained by replacing the mean and standard deviation S of the sample`s differences for each parameters.The zero in the confidence interval indicates that these two methods are not significantly different.www.ijacsa.thesai.orgAs seen in table VI, in comparison to baseline paper [1] the proposed method has a better detection rate.However, its accuracy and false alarm rate are lower.But since the confidence interval includes zero, the different is not significant.As it was mentioned, 10 executions were performed and the table also presents the mean difference and standard deviation.Due to the rarity of U2R and R2L records and existing of the imbalanced dataset, detection of these classes by using conventional data mining approaches in intrusion detection became a challenging problem.This study aims to improve the ability of intrusion detection systems in detecting U2R and R2L attacks by exploiting SMOTE and creating a boundary margin for low frequency attack classes, coupled with the CANN technique, which is a combination of classification and clustering.Moreover, this study utilizes the NSL-KDD dataset.In this study the dataset with 41 datasets is first reduced to a smaller dimensional set with 21 features using LOO method.
Subsequently, in order to evaluate the proposed method, the dataset was sampled 10 times by changing the seed of the random number generator.Furthermore, the number of U2R and R2L class instances were increased using SMOTE.The balanced dataset was then modeled by CANN and a singledimension dataset was extracted.At each execution of the algorithm, 10-fold cross validation was used for evaluations.Experimental results indicated that the proposed method outperforms the baseline approach regarding detection rate.However, it achieves lower accuracy and false alarm rate, which are not a significant difference.Results show that SMOTE coupled with CANN able to eliminate the limitation of the baseline research in detecting low-frequency attacks U2R and R2L and improves them by 94% and 50%, respectively.
1 http://www.unb.ca/research/iscx/dataset/iscx-NSL-KDDdataset.htmlpositive rate.More specifically, the classification algorithm is executed with and without each feature.If the four following conditions are satisfied, the corresponding feature is considered important and remains in the dataset. Accuracy reduction, FP reduction  Accuracy reduction, increase in FP  No change in accuracy, increase in FP  Increase in accuracy, increase in FP

Fig. 1 .
Fig. 1.Proposed algorithm Tables IV and V present the number of percentage of each class before and after sampling.

Fig. 2 .
Fig. 2. The performance of the two methods in detecting U2R and R2L attacks V. CONCLUSION

TABLE I .
FEATURES OF NSL KDD DATASET AND THEIR TYPES AND NUMBERS

TABLE III .
CONFUSION MATRIX

TABLE IV .
NUMBER OF RECORDS IN NSL DATASET

TABLE VI .
THE MEAN PERFORMANCE OF THE APPROACHES

Table
VII shows that the proposed method improves limitations in detecting U2R and R2L attacks.Moreover, figure2compares the accuracy of detecting U2R and R2L by the two approaches.