Towards a New Approach to Improve the Classification Accuracy of the Kohonen ’ s Self-Organizing Map During Learning Process

Kohonen self-organization algorithm, known as “topologic maps algorithm”, has been largely used in many applications for classification. However, few theoretical studies have been proposed to improve and optimize the learning process of classification and clustering for dynamic and scalable systems taking into account the evolution of multi-parameter objects. Our objective in this paper is to provide a new approach to improve the accuracy and quality of the classification method based on the basic advantages of the Kohonen self-organization algorithm and on new network functions to pre-eliminate the auto-detected of drawbacks and redundancy. Keywords—Artificial neural networks; self-organization map; Learning algorithm; Classification; Clustering; Principal components Analysis; power iteration


INTRODUCTION
The self-organizing map of Kohonen (SOM), is a model of artificial neural networks (ANN) widely used in different domains, particularly in the classification and clustering of multi-parameter objects.These intelligent systems are characterized by special abilities such as learning, adaptation and the possibility of visualization of Multiparameter objects with a reduced space.Other paradigms can be used, such as retro-propagation network (Counter-propagation) and Backpropagation, both types unlike the SOM map based on supervised learning algorithms that requires advance preparation of desired outputs.Our objective is to reveal the ambiguities and obstacles that may limit the application of this paradigm in the different domains of human activities, and find possible solutions to eliminate them.To reach the desired result, we made a preliminary theoretical study of the learning process, and performed experimental tests.On the basis of analysis of results achieved, we fixed the nature and details of the problem and proposed a solution already practiced and justified theoretically that can eliminate the obstacles detected.At the base of this solution, a new structure of the Kohonen network was realized.The new realization uses a block of preliminary processing of input data.The treatment method is based on a combination of two well-known algorithms which are used in the field of data analysis.The tests performed and presented show that the proposed model of the map is more advantageous to be used as a means for the creation and the application of intelligent systems in the various fields of human activities.
Except the introduction and conclusion, the work also contains four sections.The introduction presents a resume of this work, which explains the problematic and proposed solutions.In the second section, we give a detailed explanation of the problem of classification and clustering, and present examples of classical and recent methods used to solve this problem.The third section describes the process of classification and clustering using the organizing Kohonen map, also shows the detail of the learning algorithm of this paradigm of the neural network, and fixed observations of its functioning.The improved model of the map is presented in the fourth section.This part explains the detail of our follow up approach to eliminate ambiguities identified in the previous part.Also in this part, we present and explain the changes brought to the standard model map, and the functional structure of the proposed model.The fifth section contains the assessment of this work.A program was developed which is able to present the results of the map's learning on a twodimensional space, this approach allowed us to properly analyse problems in the standard model and show the advantages of the proposed model concerning the pertinent and accuracy of classification and clustering.The paper concludes by citing the observations fixed in the functioning of the standard model of the map and the advantages of the proposed model and the future prospects.

II. THE MULTI-PARAMETER OBJECTS CLASSIFICATION
AND CLUSTERING The classification problem is to establish the dependence of the input pattern (a specific object, the phenomenon or process), which is characterized by a vector of parameters (characters), to one of several predefined classes [1].www.ijacsa.thesai.orgExamples of these type of problems are handwriting recognition, language recognition, and classification of ECG signals.During clustering, the training sample with class labels is missing and performed detection of the similarity between the patterns, and the similar patterns are brought to the same category.Proximity is often understood in the sense of Euclidean distance [2,3].This problem occurs when extracting data, the study of their properties and compression.Thus, two paradigms of classification have been identified: supervised and unsupervised (clustering).
Classical supervised methods are probabilistic methods, in particular, the method based on Bayes formula adapted for manual calculations [4,5].Modern methods are not suitable for calculation without using the computers.The application of these methods produces the rules capable to through the given objects to certain specific classes.Rules solutions can be derived as characteristics probability assignment the pattern to a particular class (Bayesian method) or as a simple analytical function (discriminate analysis method).This method has certain limitations, such as lack of flexibility, which is a consequence of the application only of rules of linear solutions [6].

A. Classic structures and paradigms of learning neural networks for classifying and clustering problems
New features to classifying and clustering provide artificial neural networks (ANN).Such networks are simplified models of biological neural networks of the brain where neurons are simulated by relatively simple elements of the same type (formal neurons).Neurons are connected together by means of synaptic connections (synapses).The defining features of the ANN are distributed and parallel execution of calculations and learning opportunities.
To solve the problem of clustering ANN is used with unsupervised learning paradigm.Thus, the internal data structure is determined that is the hidden correlation between patterns.The learning process, as in the supervised learning, consists of progressive adjustment of the weighting values.Obviously, such an adjustment may be made only on the basis of information available to the neuron, that is, its status and existing weights.On this basis, and by analogy with the known principles of self-organization of the nerve cells, the learning method of Heb and the method of competitive learning [7,8] are constructed.It can be assumed that the rules for the classifying and clustering solutions are presented as a set of weights adjusted due to certain iterative procedure of learning.

B. Competitive learning
This procedure, which distributes the training set into clusters, is peculiar to the input data [9].This type of neural network processes only input vector X and thus implements the learning procedure "unsupervised".An example of neural networks with competitive learning is shown in Figure 1.All neurons in the output layer o combined with all the neurons of the input layer i connections with weight coefficients w io .When the input vector x is fed to the input of the neural network, only one output is activated ("Winner").When network correctly learns all the input vectors with the same cluster, there will be one winner.There are two methods to determine the winner and establish appropriate rules of learning: method of scalar product and the method of the Euclidean distance.The scalar product method [9]: Assume that the input vector x and weight vector w o is normalized to unit length.For each neuron in the output layer, the output value y o is calculated according to scalar product of the input vector and the vector of weighting coefficients: In the next step, output neuron k is elected with the maximum value of the output: 2 Then the value of the output neurons of the output layer is set so that y k = 1and k o y  = 0.This defines the competitive feature of the network, and the output layer can be called a layer of "winner takes all".The selection function of the output neuron with the maximum output value can be made by the neural network known as Maxnet [10].
When a neuron-winner k is elected, the weights are corrected in accordance with the formula: Where γ: coefficient of learning (rate learning).Divider shows that all the weight vectors are normalized.Note that only the weights of the winner neuron k are changing.
Changing weight coefficients according to formula (3) rotates the weight vector W O in the direction of the input vector x.Whenever vector x is supplied to the input of the network, the closest weighting vector is elected which subsequently is wrapped in the direction of the input vector.Finally, the weighting coefficient vectors rotate in the direction of areas in space where there are multiple input vectors, that is, clusters of inputs.

Euclidean distance method [9]:
In the method of scalar product, it is assumed that the input vector x and the vector of weighting coefficients w are normalized.This method may give an error while processing normalized vectors.A natural method is adapted for non-normalized input.For this, winner neuron k with the corresponding weighting vector w k is www.ijacsa.thesai.orgselected, which is closest to the input vector x, using the Euclidean distance measure: It is easy to verify that if all the vectors are normalized, then (4) reduces to (1) and ( 2).The Euclidean distance norm is this more general case of formula ( 1) and ( 2).Instead of rotation of the weight vector in the direction of the input vector being performed according to formula (3), the weight can change using the shift in the direction of the input vector: Again only the weights of a neuron winner are changed.
The important step in this recursive procedure is initializing.If the input vectors form an input set of large dimension, then initialized at random weight vector w o can never be elected as the winner and therefore, never to move and use.Therefore, usually, weight vectors are initialized set of input vectors {x}, which are randomly, selected from the input set.An improved approach to avoiding this and other problems in the competitive learning is the so-called "leaky learning" [9].The supplement changes weighting coefficients in accordance with (1.5) formula: The   '  : coefficient «leaky learning."Another similar method is competitive training, sensitive to the frequency (frequency sensitive competitive learning) [11].In this algorithm, for each neuron is memorized number of cases where he was elected as the winner.More the number of winning neurons, less susceptible will they be to competition.

III. THE PROCESS OF CLASSIFICATION AND CLUSTERING OF MULTI-PARAMETRIC OBJECTS WITH THE SELF-ORGANIZING MAP OF KOHONEN
The artificial neural networks develop rules for a welldefined resolution, which allow classifying the multiparametric objects (events, situations, processes).Unlike classical classification methods, the RNA is used to create a biological information model of the human and other beings brain [12,13].
The model of the Kohonen network uses the competitive learning method [14].This procedure distributes the objects of the learning multitude on clusters inherent to the regrouping of the input data [9].During learning, the neurons are competing, and the network fixes the winner neurons for each group of similar input objects.The fixed neurons form the centers of the clusters.The metric used in this operation is the Euclidean distance between the synaptic weights vectors, and input objects vectors.

A. The structure of the Kohonen neural network
The Kohonen network is composed of two layers: the input layer and output layerprecisely SOM.The elements of the map are dispersed in a space -usually one-dimensional or two-dimensional (Figure .2).The input data is presented as a matrix, the rows are the vectors of objects, and the columns are the components of these objects.The objects are presented to the network input one by one the idea of learning, is the application of successive rapprochement method, beginning with the random choice of the disposition of cluster centers, then the algorithm gradually improve them to perform the learning data clustering [7][8].

B. The Kohonen network learning algorithm
The learning procedure begins with the normalization of input data and synaptic weights to reduce the learning time [15].This operation is based on the following algebraic formula: Where: x i -input object component or of vector of synaptic weights; n -The number of variables in the vector x.
The main learning algorithm passes successively through a series of iterations.In each one, the learning object vectors are presented successively to network input, and the desired output is absent.At the end of this procedure, the topologically adjacent neurons respond to similar input vectors.
To fix the winner's neurons, using the metric of the Euclidean distance [4] see the formula below: 8 Subsequently the algorithm performs a correction of synaptic weights to gradually minimize the distance between the neuron winners and input objects.For this correction using the following formula [9]: Where: y i -the value of the output neuron i; w ij (t) and w ij (t +1) -the synaptic weights in iteration t and (t + 1) iterations.www.ijacsa.thesai.orgα i (t)-learning rate, this coefficient can have a value between 0 and 1, and is calculated using the following equation: 10Where: i -The iteration number; t -The iteration rate.h(d, t) -neighborhood function, it is written according to the formula below: Where: d-the distance between the winner neuron and an x neuron.δ -Constant.The learning process will be continued up to the stabilization of the self-organizing map.

C. The classification and clustering process with Kohonen's self-organizing map: The experiments results
The test is performed to fix the impact of data normalization on the learning process, and the revelation of the restrictions which may limit the right application of the network in different domains.
The data used in the test are structured to form a matrix of a dimension of (8X3), whose lines present the input vector objects.Among these objects, there are two similar objects (Input 1 and 5), four objects having linear regularities between its components (Inputs 0, 4, 2 and 6) and two other normal objects (Inputs 3 and 7) see Tab.1.
The result presented in Figure 3 shows that the network designated a winner neuron for each pair of similar objects (1, 3, 5 and 7) and there are two different winner neurons for both normal objects which seem normal.However, for linear inputs objects (0, 2, 4 and 6), the network has fixed a single winner neuron for each pair that does not conform to our wishes.The theoretical analysis and experimental tests show that the possibilities of the card in the classification and Clustering task of multi-parameter objects are limited by the loss of certain information about objects (the initial length of vectors).This loss is caused by the need of the input vectors normalization.Thus, the practice shows that the speed and learning term depend on the size and complexity of the data multitude.Our perspective is to develop intelligent systems based on neural network models with competitive learning.These systems will be characterized with news capabilities to identify and classify correctly the linear or nearly linear objects.

IV. THE IMPROVED MODEL OF THE KOHONEN SELF-ORGANIZING MAP
The proposed new map model features an input data treatment and optimization block (IDOB).The proposed model structure is represented in the figure below.This block uses an algorithm combination of two very known and approved algorithms, the principal components analysis algorithm (PCA) [16], and the iterated power method IP [17].This combination allows us to develop a method characterized by capacity derived from these two algorithms.The new skills eliminate the linear regularities between the objects components of the learning multitude, and reserve only the most informative components [18].With these capabilities, the new implementation increases the performance and accuracy of classification and clustering.

A. The input data optimization algorithm
The proposed algorithm in this section combines two very known methods: the PCA that will calculate the correlation www.ijacsa.thesai.orgmatrix, and IP which will search the Eigenvectors that constitute the rows of the resulting matrix.
The principal component analysis is a way to identify patterns in the data, and expressing data so as to highlight their similarities and differences.The main advantage of the (PCA) is that once these patterns are found in the data, it can compress data by reducing the number of dimensions, without losing too much information.This technique is the basis of our method to achieve our objectives referred to in this work.
The initial data is organized as an array, whose rows are the individuals (objects) and columns present variables (Components) (see Figure5).  is the covariance of the variables k and l, and is the variance of the variable k.
In the last step and to develop the correlation matrix R, we calculate the two diagonal matrices as a function of V for the following way.The correlation matrix R comprises the coefficients of linear correlation between p variables taken two by two.It sums and shows the structure of linear dependencies between p variables.It is symmetrical and the diagonal is composed of 1. R is written according to the following way of V:

4.7
Where: D 1/s is the diagonal matrix containing on its diagonal.
After calculating the matrix R using the iterated power method, the Eigenvectors are searched that constitute the rows of the resulting matrix of input objects.

[ ] 20
The figure below presents the proposed algorithm flowchart.In the diagram, the first seven steps of the method ACP calculate the correlation matrix R. The next two steps of the iterated power method use this matrix to search the Eigenvectors and develop the reduced final matrix.

B. The classification and clustering process with improved Kohonen's self-organizing map: The experiments results
To test and verify the capabilities of the model proposed in comparison to the standard model, the experimental test in this section is based on the same data used to test the standard version, and with the same conditions.Thus, all the necessary modules have been added to our application, for which works in two modes: standard and modified.The figure below shows the result of the test performed.www.ijacsa.thesai.orgThe data in the figure above shows that the proposed model gave the desired results, and it has correctly classified the input objects.Based on these results, it is observed that the new model has designated seven winner neurons, unlike the standard model that has repartee the multitude of data only on five Clusters.Thus, we note that the IDOB has reduced the data size, and each object was described only by one component.

V. EVALUATION
To evaluate the innovations brought in this work and make conclusions, a graphical presentation of the input data figured in Table .2 is used and the results in the figures are given below.The length of the vectors used in this example (two components), has been adjusted to have the possibility of presenting input data into a two-dimensional space, which facilitates the understanding of the problems and improves results interpretation.The graphical presentation shows that the four input objects are correctly dispersed in the realization space (Figure 8.A), but after the normalization processing we can compress the realization space but the vectors lose information about their initial lengths (Figure 8B).The loss of this information has prevented the standard model of the card correctly classify and clustering the data.Figure 8.C shows that after learning, the expected result was not obtained and the SOM has designated only two winner neurons for the four input objects.In the second graphical presentation (Figure 9), we interpret the input data and results for the proposed model of the map.Part A of the figure shows that the IDOB has well filtered and reduced the initial data and the new dimension of the matrix formed by the block is (4x1).Thus, the three parts of the figure showing how the proposed model have eliminated the regularities between the components of objects and the learning algorithm has successfully brought the winners neurons to input objects.
The software developed for both map models (standard and modified); visualize the results on two-dimensional maps especially for the winners and their clusters (see both figures below).Map as a grid of rectangles corresponding to the neurons in the Kohonen layer, the index of the neuron in this layer is the same for the rectangle on the map.
To determine the winner neurons and Clusters on two maps the application uses the coloring technique.The rectangle that presents the winning neurons on the map will have a unique color; its neurons represent the centers of clusters in the second map.In the second map, the rectangles that present the same Cluster neurons will be colored with a unique color.According to the two figures, we can say that the proposed model exceeds the standard model concerning the accuracy of the classification.The two maps in Figure 10 show that the standard model has designated five winner neurons (five Clusters center) and five clusters, but maps in Figure 11 show that the proposed model has fixed seven centers of clusters according to the expected result.

VI. CONCLUSIONS AND PERSPECTIVES
On the basis of evaluations and observations described above we arrive to the following conclusions and perspectives: 1) The Kohonen network derived the animal brain functionality, and can be used to realize the intelligent systems.
2) The intelligent systems based on this paradigm are characterized by special abilities such as learning, adaptation and the possibility of visualization of Multiparameter objects with a reduced space.
3) The normalization procedure of input objects has advantages, but it prevents the card correctly classify linear or nearly linear objects.
4) The proposed method is approved theoretically.This method has eliminated the regularity problems between the components of linear objects and reduces the dimension of data.
5) The data processed by the blocks IDOB will be deformed, which complicates their analysis.
6) Despite its advantages, the proposed model of the card must beings improved to solve the problem of the deformation of data, and increase the chance of its application in the different activity domains.

Fig. 3 .
Fig. 3. Learning results of the self-organizing map Matrix 8X3

Fig. 4 .
Fig. 4. Structure of the new model of the Kohonen map

Fig. 5 .
Fig. 5.The initial array dataIn the first step, the algorithm calculates the vector of main point g.This point is the center of the points cloud in a space F. See the formula below.
the base of the vector g is calculated the data centered matrix, which is written in terms of X of the following way: 15 Where: g t is the transposed of g.The term centered signifies that the means of the variables ̅ are zero.The centered data matrix Y is used in this step for calculating the variance-covariance matrix V, which is written according to the following way of Y:16 Where: Y t is the transposed of Y.The V matrix is presented as follows:

Fig. 7 .
Fig. 7. the learning results of the improved model of map (Matrix 8X3)

Fig. 8 .
Fig. 8. Graphical presentation of data and the learning outcome of the SOM card (Classic Model)

Fig. 9 .
Fig. 9. Graphical presentation of input data and the SOM learning result (Improved Model)

Fig. 10 .
Fig. 10. the standard SOM, The results of the classification and clustering for the matrix (8 x 3)

Fig. 11 .
Fig. 11.The modified SOM, The results of the classification and clustering for the matrix (8 x 3)