Investigating the Effect of Different Kernel Functions on the Performance of SVM for Recognizing Arabic Characters

A considerable progress in the recognition techniques of Latin and Chinese characters has been achieved. By contrast, Arabic Optical character Recognition is still lagging in spite that the interest and research in this area is becoming more intensive than before. This is because the Arabic is a cursive language, written from right to left, each character has two to four different forms according to its position in the word, and several characters are associated with complementary parts above, below, or inside the character. Support Vector Machines (SVMs) are used successfully for recognizing Latin, and Chinese characters. This paper studies the effect of different kernel functions on the performance of SVMs for recognizing Arabic characters. Eleven different kernel functions are used throughout this study. The objective is to specify which type of kernel functions gives the best recognition rate. The resulting kernel functions can be considered as base for future studies aiming at enhancing their performance. The obtained results show that Exponential and Laplacian Kernels give excellent performance, while others, like multi-quadric kernel, fail to recognize the characters, speciallywith increased level of noise. Keywords—SVM; Kernel Functions; Arabic Character Recognition


INTRODUCTION
Optical character recognition systems improve humanmachine interaction and are urgently required for many governmental and commercial organizations.A considerable progress in the recognition techniques of Latin and Chinese characters has been achieved.By contrast, Arabic Optical character Recognition (AOCR) is still lagging because of the special characteristics of the Arabic Language.It is a cursive language, written from right to left, each character has two to four different forms according to its position in the word, and most characters are associated with complementary parts above, below, or inside the character.However, the interest and research in this area is becoming more intensive than before.
SVMs is a powerful tool compared to other supervised classification techniques.It is based on statistical learning theory developed by Vladimir Naumovich Vapnik [1] back in 1963 and since then, his original ideas have been perfected by a series of new techniques and algorithms including that of Olusayo D. Fenwa et al. [2], who evaluated the Performance Of PSO-Based Kernel Support Vector Machine in Offline Digit Recognition.One of the most important design choices for SVMs is the kernel-parameter, which implicitly defines the structure of the high dimensional feature space where a maximal margin hyperplane will be found.However, before this stage is reached in the use of SVMs, the actual kernel must be chosen, as different kernels may exhibit different performance.
This paper investigates the effect of eleven different kernels on the performance of SVMs in recognizing Arabic characters.The objective is to extract the kernels giving the best performance.Future work can then be elaborated for enhancing the performance of these kernels.The rest of the paper is organized as follows.Section II reviews the literature.Section III describes the used algorithm.The analysis and discussion of the obtained results are summarized in Section IV. Conclusions of this work with some future lines of research are presented in the last section.

II. LITERATURE SURVEY
Support Vector Machine (SVM) is a state-of-the-art classification method that belongs to the general category of kernel methods.A kernel method is an algorithm that depends on the data only through dot-products.When this is the case, the dot product can be replaced by a kernel function which computes a dot product in some possibly high-dimensional feature space.This approach has two advantages: First, the ability to generate nonlinear decision boundaries using methods designed for linear classifiers.Second, the use of kernel functions allows the user to apply a classifier to data that have no obvious fixed-dimensional vector space representation.
In recent years, Kernel methods have received major attention, particularly due to the increased popularity of the Support Vector Machines.Kernel functions can be used in many applications as they provide a simple bridge from linearity to non-linearity for algorithms which can be expressed in terms of dot products [3].
A linear support vector machine is composed of a set of given support vectors y and a set of weights w.The computation for the output of a given SVM with N support vectors y 1 , y 2 , … y N and weights w 1 , w 2 , … , w N is then given by: Using kernels, the original formulation for the SVM given SVM with support vectors y 1 , y 2 , ….. , y N and weights w 1 , w 2 , … , w N is now given by: This work investigates the effect of eleven different kernels on the performance of SVM for recognizing Arabic characters.The most popular kernels for real-valued vector inputs are shown below [4]: 1) Linear (trivial) Kernel: Linear kernel, the simplest kernel function, is given by the inner product <x,y> plus an optional constant c.Algorithms using a linear kernel are often equivalent to their non-kernel counterparts.
where, c and S are kernel-specific parameters (5)

4) Multiquadric Kernel
The Multiquadric kernel can be used in the same situations as the Rational Quadratic kernel.

6) Polynomial Kernel
The Polynomial kernel is a non-stationary kernel that is well suited for problems where all the training data is normalized.
Adjustable parameters are the slope α, the constant term c and the polynomial degree d.

7) Gaussian Kernel
The Gaussian kernel is an example of radial basis function kernel.
The adjustable parameter б plays a major role in the performance of the kernel, and should be carefully tuned to the problem at hand.

8) Exponential Kernel
The exponential kernel is closely related to the Gaussian kernel, with only the square of the norm left out.It is also a radial basis function kernel.

9) Laplacian Kernel
The Laplace Kernel is completely equivalent to the exponential kernel, except for being less sensitive for changes in the sigma parameter.Being equivalent, it is also a radial basis function kernel.
It is important to note that the observations made about the sigma parameter for the Gaussian kernel also apply to the Exponential and Laplacian kernels.

10) Hyperbolic Tangent (Sigmoid) Kernel
The Hyperbolic Tangent Kernel is also known as the Sigmoid Kernel and as the Multilayer Perceptron (MLP) kernel.
It is interesting to note that a SVM model using a sigmoid kernel function is equivalent to a two-layer perceptron neural network.This kernel was quite popular for support vector machines due to its origin from neural network theory.There are two adjustable parameters in the sigmoid kernel, the slope alpha and the intercept constant c.A common value for alpha is 1/N, where N is the data dimension.

11) Multi-Layer Perceptron:
The long established MLP, with a single hidden layer, also has a valid kernel representation.
Many authors tried the investigation of using SVMs and similar tools for recognizing Arabic characters and categorizing Arabic text.Mahmoud Zennaki et al., in [5], presented a comparative study of SVM models for learning handwritten Arabic Characters.Eugen-Dumitru Tautu and Flrin Leon, [6] examined the effect of 4 kernels on the performance of SVM for recognizing English handwritten characters.The examined kernels are: linear, RBF, polynomial, and sigmoid functions.They found that the type of the kernel function affects the recognition accuracy.Behjat Siddiquie et al, [7] tried combining multiple Kernels for efficient image classification.S.F.Bahgat et al, [8] proposed a Hybrid Technique for Recognizing Arabic Characters.

III. PROPOSED APPROACH
The process starts with creating the database of the Arabic alphabetic character images used throughout the study.The database contains the character images, the feature vectors of noise-free images, as well as the feature vectors of character images corrupted by additional salt-and-pepper noise with levels ranging from 0.1 to 0.9.The SVM is first trained using the noise-free feature vectors for each used kernel.The SVM is then used for the classification of noisy character set for each kernel.A comparative analysis is then carried out to conclude which kernels are suitable and which are not.A detailed description of the used algorithm is as shown below.

INPUT:
Noise free database for the Arabic 28 character images.OUTPUT: Recognition rate of Arabic characters using 11different kernels.

IV. RESULTS ANALYSIS AND DISCUSSION
This section investigates the performance of SVMs in recognizing the Arabic characters, corrupted by salt and pepper noise with levels starting from 0.1 to 0.9, using eleven different kernels.Samples of the free and noisy Arabic character images used in this study are shown in Fig. 1.
As shown in TABLE I and Fig. 2., it is clear that Laplacian, Exponential, Gaussian, Inverse Multiquadric, and Rational Quadratic Kernels give excellent results for all noise levels.On the other hand, the multi-quadric kernel gives very bad results for all noise levels.Linear, Quadratic, Polynomial, Hyperbolic Tangent, and Multilayer Perceptron Kernels, have monotonically decreasing performance with increasing the noise level.
Excluding the Multiquadric kernel, TABLE II and Fig. 3. show that there are two groups of kernels according to their performance.Group 1, which has the best performance for all noise levels, includes Laplacian, Exponential, Gaussian, Inverse multiquadric, and Rational Quadratic Kernels.Group 2, which has a monotonically decreasing performance with increasing noise levels, includes Linear, Quadratic, Polynomial, Hyperbolic Tangent, and Multilayer Perceptron Kernels.Focusing on the group having the best performance, TABLE III and Fig. 4., show that the exponential and Laplacian kernels give the best performance in the set of the examined kernels.However, there are another 14 kernels that will be examined to extract the final result.

V. CONCLUSION
SVMs are used as a classification tool for the recognition of Arabic characters.However, before this stage is reached in the use of SVMs, the actual kernel must be chosen, as different kernels may exhibit different performance.This paper studies the effect of eleven different kernel functions on the performance of SVMs for recognizing Arabic characters.
The obtained results show that Laplacian, Exponential, Gaussian, Inverse Multiquadric, and Rational Quadratic Kernels give excellent results for all noise levels.On the other hand, the multi-quadric kernel gives very bad results for all noise levels.Linear, Quadratic, Polynomial, Hyperbolic Tangent, and Multilayer Perceptron Kernels, have monotonically decreasing performance with increasing the noise level.Further investigation showed that Exponential and Laplacian kernels give the best performance.Future work will stress on another set of kernels to extract the most suitable kernel functions for recognizing Arabic characters.

Fig. 1 .
Fig. 1.Samples of the used database

Fig. 2 .Fig. 3 .Fig. 4 .
Fig. 2. Recognition rate of Arabic characters using SVM with different kernels as a function of salt & pepper noise

TABLE I .
RECOGNITION RATE OF DIFFERENT KERNELS (%) IN THE PRESENCE OF SALT AND PEPPER NOISE

TABLE III .
KERNEL FUNCTIONS SHOWING PERFORMANCE ABOVE 90%