Logarithmic Spiral-based Construction of RBF Classifiers

Clustering process is defined as grouping similar objects together into homogeneous groups or clusters. Objects that belong to one cluster should be very similar to each other, but objects in different clusters will be dissimilar. It aims to simplify the representation of the initial data. The automatic classification recovers all the methods allowing the automatic construction of such groups. This paper describes the design of radial basis function (RBF) neural classifiers using a new algorithm for characterizing the hidden layer structure. This algorithm, called k-means Mahalanobis distance, groups the training data class by class in order to calculate the optimal number of clusters of the hidden layer, using two validity indexes. To initialize the initial clusters of k-means algorithm, the method of logarithmic spiral golden angle has been used. Two real data sets (Iris and Wine) are considered to improve the efficiency of the proposed approach and the obtained results are compared with basic literature classifier Keywords—Radial Basis Function neural network; classification; k -means; validity index of Davis Bouldin; Mean Squared Error; Mahalanobis distance; Logarithmic spiral; golden angle; golden ratio


INTRODUCTION
Clustering is one of the most useful tasks in data mining process for discovering groups and identifying interesting distributions and patterns in the underlying data.The problem with Clustering is about partitioning a given data set into groups (clusters) such that the data points in a cluster are more similar to each other than points in different clusters [1].
In recent years, a number of clustering algorithms has been proposed and is available in literature.The radial basis function (RBF) neural network is one of the most used in data classification.
RBF neural networks consist of three layers: an input layer, a hidden layer and an output layer.The input layer corresponds to the input vector feature space and the output layer corresponds to the pattern classes [2].So the whole architecture is fixed only by determining the hidden layer and the weights between the middle and the output layers [3].
Its training procedure is usually split into two successive steps: training in the hidden layer followed by training in the output layer [4].First, the centers of the hidden layer (HL) neurons are selected by clustering algorithms such as k-means [5], [6], support vector machine (SVM) [7] or hierarchical clustering [8].Second, the weights connecting the hidden layer with the output layer are determined by Singular Value Decomposition (SVD) or by Least Mean Squared (LMS) algorithms.
One of the used techniques to find the optimal number of this HL is the logarithmic spiral which has seen a significant amount of research on nature-inspired optimization techniques such as neuro-computing in the past 25 years, evolutionary and genetic algorithms, particle swarm optimization.Most recently, a new multipoint meta-heuristics research method has emerged for 2-dimensional continuous optimization problems based on the analogy of spiral phenomena in nature, called 2-dimensional spiral optimization first proposed by Tamura and Yasuda in 2010 [9].
Focused spiral phenomena are approximated to logarithmic spirals, which frequently appear in nature, such as whirling currents, nautilus shells and arms of spiral galaxies.A Two-dimensional spiral optimization uses the feature of Logarithmic Spirals [LS] [9].
In this paper, a new learning algorithm is proposed for the construction of the radial basis function networks solving classification problems.It determines the proper number of hidden neurons automatically and calculates the centers values of radial basis functions.After the selection of the hidden neurons, the widths of nodes are determined by the P-nearest neighbors heuristic, and the weights between the hidden layer and the output layer are calculated by the pseudo-inverse matrix.
The aim of this approach consists in transforming the problem of determining the number of hidden layer neurons to a clustering problem.In order to determine the number of clusters in the data of each class, the k-means algorithm is combined with two different validity indexes (the validity index of Davis Bouldin for the first classifier and Mean Square Error for the second classifier).
In k-means algorithm, the used distance corresponds to the Mahalanobis distance.A solution is also given to overcome www.ijacsa.thesai.org the problem of setting the start values for the initial centers needed to start this algorithm using the proposed method "The logarithmic spiral golden angle".Two different real databases are used in order to evaluate the two proposed classifier performances.
Next section presents the problem of the construction of the hidden layer for RBF neural networks.Section 3 describes in detail the logarithmic spiral golden angle.Section 4 is devoted to the elaborated solution to overcome the problem of the k-means algorithm.Section 5 describes the construction of the two proposed RBF classifiers.Experiments and discussions are presented in Section 6, followed by concluding remarks in Section 7.

II. PROBLEM STATEMENT OF USE
The construction of the hidden layer of RBF neural networks is by clustering algorithms such as k-means.Kmeans clustering algorithm is one of the best-known algorithms used in clustering.
However, it still has some problems one of which is in its initialization step which is generally done randomly by users.Another disadvantage of k-means is that it converges to local optimum, depending on its random initialization.
The k-means algorithm classifies objects to a pre-defined number of clusters, which is given by the user (assume k clusters).The idea is to choose random cluster centers, one for each cluster.These centers are preferred to be as far as possible from each other.The Starting points affect the clustering process and results [10], "Fig.1".
Each boot (initialization) is a different solution (local optimum) which can in some cases be far from the optimal solution (global optimum) [11].A simple solution to this problem is to run the algorithm several times with different initialization and retain the best combination found.
The use of this solution is limited because of its cost and because of the possibility of finding better results in a single execution [12].The main idea of this work is to improve the performance of the k-Means clustering algorithm by fixing its weaknesses.Randomness is one of the techniques wildly used in initializing K-means algorithm that is why it is considered as the main point of weakness that should be solved.
However, because of the sensitivity of k-means to its initial points, two solutions have been proposed to this problem.The first one is to initialize the centers of k-means algorithm using the circle method [13].The obtained results of this algorithm are the initial centers positions k C represented by the "Fig.2".
The second solution is to initialize the centers of k-means algorithm using the logarithmic spiral golden angle in order to improve the clustering performance.In this paper, the second solution will be explained.

III. THE LOGARITHMIC SPIRAL GOLDEN ANGLE
The logarithmic spiral golden angle is a specific case of the logarithmic spiral.It represents a plane curve centered in a starting point and parameterized by the radius r , the angle  and the Golden Ratio  .

A. The logarithmic spiral
A Logarithmic Spiral is a plane curve for which the angle between the radius vector and the tangent to the curve is a constant [14].Such spirals can be approximated mathematically defined by the following equation on the 2dimensional polar coordinate system   , r  as [9]: In this work, the factor b of the logarithmic spiral has been set to zero ( 0 b  ), it goes back to simplify the polar radius as follows: The following equation of the logarithmic spiral golden angle is obtained:

B. The Golden ratio
The irrational number, golden ratio is also known as golden section by the ancient Greeks, golden proportion, divine proportion or golden number [15].
The golden ratio  , has many properties which people are eager to know.It is a number that is equal to the reciprocal of its own with the addition of 1: Likewise, the ratio of any two consecutive Fibonacci numbers converges to give approximates of 1.618, or its inverse, 0.618.This shows the relationship between Fibonacci numbers and golden ratio [16].
If the possibility of dividing a line in such a way that the ratio of the whole length to the length of the longer segment happens to be equal to the ratio of the length of the longer segment to the length of the shorter segment, then it could be said that the ratio is a golden ratio [15], "Fig.3".

C. The Golden angle
In geometry, the golden angle is created by dividing the circumference of a circle c in two sections, a longer arc of length a and a smaller arc of length b such that: c a b 

IV. PROPOSED INITIALIZATION OF THE K-MEANS ALGORITHM WITH LOGARITHMIC SPIRAL GOLDEN ANGLE
The k-means algorithm aims to minimize the distance between the object and the center of its group.In this section, the k-means algorithm based on the Mahalanobis distance and its proposition for initialization of the centers are presented.

A. The k-means algorithm Mahalanobis distance specifications
There are different types of distances such as: Minkowski distance, the average, the family of metrics, Euclidean Weighted and the Euclidean distance which is the most used, e.g.applied in the RBF Networks.[17] Moreover, the Mahalanobis distance is a distance measure and its utility is a way to determine the similarity between two multidimensional random variables.It differs from Euclidean distance, because the Mahalanobis distance takes into account the correlation between random variables, [17].The Mahalanobis distance is defined by:

 
Cov X is the covariance matrix.
If the elements x and y are independent, the covariance matrix is the identity and the Mahalanobis distance is equal to the Euclidean distance.The proposed algorithm based on the combination of Mahalanobis distance with k-means is described by the following steps: www.ijacsa.thesai.org Step 1: -Determine the size N of the data base of X .
-Determine the number k of centers to be used in the observation space C .
-Initialize the vector of the new positions of the centers * C to zero.
Step 2: -Determine the covariance matrix

 
Cov X with the following equation:

End While End
To increase the performance of the k-means algorithm Mahalanobis distance, a solution is proposed to initialize the k-means algorithm using the logarithmic spiral golden angle parameterized by the radius r , the angle  and the Golden ratio  and the Golden angle  [18].This solution is divided into several steps: The first step is to calculate the maximum distance between two individual points   , ab belonging to the database, then to define the middle ground G between these two individual points and determine the radius R Gb  "Fig.5".k , the suggestion of Bezdek was adopted [19] as follows : max kN  ,( N is the size of the database) .
The forth step is to determine the positions of the centers of the logarithmic spiral golden angle with center G and radius ra  "Fig. 6".The calculation of the center position k C is performed by applying the following formula: cos( ) sin( )  -Fix the increment of the angle  by the factor   -Fix the increment of the radius r by the factor max max Step 4: -Determine the positions of the centers belonging to the logarithmic spiral golden angle with center G and radius ra  according to the following formula: cos( ) sin( )

B. Evaluation Measures
Using an unsupervised clustering algorithm, such as kmeans algorithm, requires the determination of the number k of groups leading to the execution of the algorithm repeatedly for different values of this parameter.
For optimal number of groups, a criterion should be used to evaluate the result of the algorithm.This criterion is known as the validity index [1], [20]- [22], [15, 16, 17, and 18] name based on the notions of compactness and separation.
In literature, there are a lot of validity indexes, most of them are based on the notions of compactness within different groups and the separability between these different groups.In this article, the Davies-Bouldin index and the Mean Squared Error will be used as two validity indexes of neural classifiers.

C. Davies-Bouldin Index
This index takes into account both of the compactness and the separability of groups [23].Its value is much lower than the groups are compact and well separated.It promotes hyperspherical groups and is, therefore, particularly wellsuited for a use with the k-means algorithm.The DB I index is defined by the following expression: Where   ci dcis the average distance between an object and its group i c following the center and

 
, cc i j D c c is the distance between the centers of groups i c and j c with:

D. Mean Squared Error
The Mean Squared Error is frequently used to assess the risk of an estimator.It is also useful to relay the concepts of bias, precision, and accuracy in statistical estimation.In this work, the MSE was used for groups' compactness measure for each centroid [24]; it is notably equivalent to the Euclidean function of the k-means algorithm: With:

NEW ALGORITHMS FOR THE CONSTRUCTION OF THE HIDDEN LAYER OF THE RBF CLASSIFIER
Two new algorithms were used to characterize the hidden layer classifier i.e. to determine the number of centers of different Gaussian and the value of each center.
In what follows, the principle of the proposed classifiers is presented.It is also explained how the two validity indexes (IDB and MSE) are combined with the k-means algorithm Mahalanobis distance to determine automatically the number k of groups.
However, it is necessary to fix a maximum number of centroids max k .The max k value can be defined by the user if he/she knows the structure of his database.Given that it is not always the case, the Bezdek [19] suggestion is adopted, so the max kN  ( N is the size of the database) is chosen.
Applying these algorithms to all classes and summing the number of the obtained groups, the number of neurons in the hidden layer is determined.A neuron is then assigned to each group.For this RBF classifier, the database was partitioned   Principal component analysis (PCA) is a widely used statistical technique for unsupervised dimension reduction.Kmeans clustering is a commonly used data clustering for performing unsupervised learning tasks [25].
The PCA is based on the calculation of averages, variances and correlation coefficients.The main basis of dimension reduction is that PCA picks up the dimensions with the largest variances.
The PCA allows, in the same time, a reduction of data and an easier interpretation in the treated domain, as the new dimensions are often very significant [26].In this case, the two largest variances were chosen to represent the new database.To complete the construction of the hidden layer classifier, there is a second parameter to consider in the neurons, which is the width factor j  for each centroid j c ( 1,..., jk  ).This factor is calculated using the following formula: Step 4.3: -Calculate the compactness and separability of groups with the Davies-Bouldin Index: DB I .
Step 4.4: -Save DB I variable in the table called _ Tab IDB .
Step 5: -Determine  : the DB I lowest index of the table _ Tab IDB .
-Take the following centers positions as the optimal number required classifying 1  class centers.End

B. Construction of RBF classifier KMD-LS-MSE
The proposed algorithm (KMD-LS-MSE) based on the kmeans algorithm with Mahalanobis distance combined with the Mean Squared Error validity index.This classifier determining the number and the centers values of the hidden layer for each class of the database is described below: www.ijacsa.thesai.org-Determine the size n and the number of characters (the attributes) p of the data base 1 X  .
Step 2: -Initialize the minimum number of centroids min 2 K  and then look for the maximum number of centroids by max Kn  .
-Initialize the variables min a k K  and 1 d  .
Step 3: -Apply Algorithm Init_Centres_Kmeans_ Logarithmic_Spiral which initializes the centers for kmeans algorithm of the data base 1 X  .
Step 4: -Repeat the following steps until Step -Take the following centers positions as the optimal number required classifying 1  class centers.End

C. Calculation of synaptic weight
After determining the parameters of the proposed classifier hidden layer, the learning is finished by the calculation of the synaptic weight ij w , connecting the hidden layer neurons to those of the output layer.
The linearity property of the outputs () jl yxof the network is used.The expression of each of the m outputs is written as: ( ) ( ) The global output of the network is written as follows: Y H W  The objective is to determine the matrix W that minimizes an error function, chosen as the square of the sum of classification errors.
The weight of the output layer can be calculated by the following matrix equation: With H : The matrix of Radial Basis Functions Y : The matrix of the output layer.
W : The weight matrix of centroids.ij  : The Gaussian widths of the matrix H .
The above equation is giving by: 1 Given that the H matrix is rarely square, the pseudoinverse of the matrix H is applied according to equation ( 15):

VI. EVALUATION OF RBF CLASSIFIERS
The purpose of this section is to evaluate the performance and efficiency of the proposed RBF neural classifiers.The performance of these RBF neural networks classifiers is tested with two different databases: Iris and Wine among the different data sets available from the machine learning community by the University of California at Irvine (UCI) [27].
The first test is carried out with the Wine database which includes the results of a chemical analysis of types of wine produced in different regions of Italy from different grapes varieties.The concentration of 13 components are indicated for each of the 178 types of wine (patterns) which are analyzed and divided into three classes (59 in Class 1, 71 in Class 2 and 48 in Class 3).
The second test is done with Iris database which is one of the most popular data set to examine the performance of novel methods in pattern recognition and machine learning.It is composed of three classes (i.e., iris Setosa, iris Versicolor and iris Virginica) each having 50 patterns with four features.
To evaluate the proposed classifiers performances, the holdout method is used.It consists on dividing the initial data into two independent sets: one for training and the other for testing the classifier performances.
The results given by the RBF classifier built with this algorithm are compared with those obtained with other neural www.ijacsa.thesai.orgclassifiers: the Learning Vector Quantization (LVQ) classifier proposed by Kohonen, the RBF neural network classifier for which the hidden layer is obtained using adaptive Pattern Classifier (APCIII) [28], the Multi-Layer Perceptrons classifier (MLP) and with a reference: the K nearest Neighbor (KNN).
The present comparative results of different classifiers over Wine and Iris are illustrated in Table I   Considering Wine database, the best recognition rate is obtained by the KMD-LS-IDB proposed classifier and then the proposed classifier KMD-LS-MSE.For Iris database, the best recognition rate is given for the KNN classifier; however, the difference with the two proposed classifiers is not important.
Then, the proposed algorithms give good results in terms of recognition rate but the most powerful of them is the classifier KMD-LS-IDB.

VII. CONCLUSION
In this paper, new RBF neural networks classifiers has been designed to classify database.The proposed algorithms aim to deduce the centers of the hidden layer neurons and to calculate the number of the neurons in particular.
The basic idea of this approach is to select the training data from the database class by class and to decide about the optimal number of neurons in each class by using two different validity indexes (the validity index IDB and the MSE).This number is integrated in the k-means algorithm with the Mahalanobis distance.
A solution was also proposed to overcome the problem of initialization of centers necessary for the start of the K-means algorithm using the method of the logarithmic spiral golden angle.
The obtained classifiers results are satisfactory in comparison with other considered classifiers in literature for two real databases (Iris and Wine).

Fig. 2 .
Fig. 2. Tracing of initialization of the k centers maximum on the outline of the circle of radius r and spaced by the angle  can be transformed into Cartesian coordinates as follows:

Fig. 3 .
Fig. 3. Dividing of a whole length AC into two segments AB and BC This gives mean ratio if AB AC BC AB  .If the value of AB is set to be x , and use 1 to represent the length of BC, then 1 1 xx x   is obtained.Then the irrational number is the only positive solution of the equation 2 10 xx    , so 15 2 x   Its value is: 15 1,6180339887 2    .Where the Greek letter phi (  ) represents the golden ratio.

Fig. 4 .
Fig. 4. Golden angle measurement The angle formed by the arc b of circle c is called the golden angle .It derives from the golden ratio  .  1 2,391 rad 2 2 2 137,5

.N
Step 4: -Recalculate the position of each new center: = the set of points belonging to the center j c and 1,..., jk  .
The second step is to calculate the golden number  by applying the following formula:15 1,6180339887 2  The third is to initialize the values of the logarithmic spiral golden angle on the polar coordinate system   ,

Fig. 5 .
Fig. 5. Tracing of the two most distant individuals   , aband their medium

Fig. 6 . 1 :Step 2 :Step 3 :
Fig. 6.Tracing of the initialization of the k centers maximum on the outline of the logarithmic spiral golden angle Thereafter, the positions of these max k centers in the variable The next step is to determine the number of centers and the center position of each class matrix are grouped and the k-means algorithm Mahalanobis distance is applied to the new positions of the centers

1 : 2 K-
training dataA.Construction of the RBF classifier KMD-LS-IDBThe proposed algorithm (KMD-LS-IDB) based on the kmeans algorithm with Mahalanobis distance combined with the Davies-Bouldin validity index.This classifier determining the number and the centers values of the hidden layer for each class of the database is described below: base d X , taking the case of the block.(The same approach for other classes).Output: -The position of each center -Determine the size n and the number of characters (the attributes) p of the data base 1 X  .Step 2: -Initialize the minimum number of centroids min  and then look for the maximum number of Init_Centres_Kmeans_ Logarithmic_Spiral which initializes the centers for kmeans algorithm of the data base 1 X  .Step 4: -Repeat the following steps until max Deduce the number of the centers k .End If Step 4.2: -Apply k-means algorithm with Mahalanobis distance to determine the new positions of the base d X , taking the case of the block.(The same approach for other classes).Output: -the position of each center

2 :
-Apply k-means algorithm with Mahalanobis distance to determine the new positions of the and TableII.

TABLE I .
RESULTS OF THE RECOGNITION RATE OVER WINE DATABASE