An Improved Particle Swarm Optimization Algorithm with Chi-Square Mutation Strategy

Particle Swarm Optimization (PSO) algorithm is a population-based strong stochastic search strategy empowered from the inherent way of the bee swarm or animal herds for seeking their foods. Consequently, flexibility for the numerical experimentation, PSO has been used to resolve diverse kind of optimization problems. PSO is much of the time caught in local optima in the meantime taking care of the complex real-world problems.Considering this, a novel modified PSO is introduced by proposing a chi square mutation method. The main functionality of mutation operator in PSO is quick convergence and escapes from the local minima. Population initialization plays a critical role in meta-heuristic algorithm. Moreover, in this work, to improve the convergence, rather applying random distribution for initialization, two quasi random sequences Halton and Sobol have been applied and properly joined with chi-square mutated PSO (Chi-Square PSO) algorithm. The promising experimental result suggests the superiority of the proposed technique. The results present foresight that how the proposed mutation operator influences on the value of cost function and divergence. The proposed mutated strategy is applied for eight (8) benchmark functions extensively used in the literature. The simulation results verify that Chi-Square PSO provide efficient results over other tested algorithms implemented for the function optimization. Keywords—Particle Swarm Optimization; Chi-Square Mutation; Population Initialization.


I. INTRODUCTION
The term "swarm intelligence" is practiced as to explain the algorithm and distributed problem solvers, motivated by the common actions of colonies of insects and other animal groups.Swarm Intelligence (SI) based systems are normally buildup of simple agents of population that are communicating internally with each other and with their environment [1].
Likewise, other evolutionary algorithms (EAs), the particle swarm optimization (PSO) technique is a population based metaheuristic search approach that devised from nature aspect.Such type of approaches commonly needs extra objective function evaluations that compared with gradient search techniques.These techniques offer stunning features like easiness in the numerical implementation for both discrete and continuous optimization problems and more powerful solution creations for seeking the global solutions.PSO algorithm seeks the optimum solution inside the population called as flock or swarm.PSO avails the advantage from two type of training: cognitive training focused on particle's own history while social training concentrated on swarm history of information sharing collected from all the individuals of the swarm.
Kennedy and Eberhart formally introduced PSO, it got enticed notable attraction in last decade.Vast majority of research on this subject is concerned either with the mathematical analysis or to enhance the algorithm for attaining the quicker, robust, and scalable candidate solutions.Main motivation of the posterior, stuck in local optima by the solution or premature convergence.An EA is trapped to the local optima, if it is not capable to investigate all the search space except the explored region, and another area persists that hold a best solution better than to the currently find solution.One of the major causes for the deficiency of diversity is premature convergence.
Diversity is vital important for vigorous searching in the given search space while the mutations are fundamental operators to give the dynamic diversity inside the swarm [2].Besides the designing of new mutation operator, researchers have placed fewer efforts to explore that how to use the mutation operator [3] and to find what type of diversity supposed to be available in the swarm.Hence, after the detail exploration of diversity concept focused on qualification and quantification studies, this paper presents new mutation strategy and operator to give useful diversity in the swarm.The new proposed technique has been used on the selected benchmark functions.From these test cases, it is shown that proposed technique has given the better results than other variants of PSO.The core objective of this strategy was to find which particle should be mutated and when; it should be mutated.The proposed technique also gives controlled diversity in the swarm.
The rest of the paper is structured like this: Section 2 reviews the technical background of PSO algorithm where Standard PSO is discussed in Section 3. Section 4 carries the proposed algorithm.In Section 5, the computational simulation results and comparison are carried out.Conclusion and future work are presented in Section 6.

II. LITERATURE REVIEW
PSO might have some problems associated to rate of convergence, premature converged solution, poor accuracy and failure of diversity.Several modifications have been introduced till now to overcome these issues [4].Improved approaches can split into two types concentrating on improvements of hardware and software.The hardware is concerned with parallel computing.While the software improvements can be more categorized like integrating with other heuristic techniques, motivation from other stochastic methods, manipulating and restructuring of velocity update equation, managing the neighborhood topology, tuning the parameters during the update phase and swarm size.
PSO has been incorporated with other metaheuristic algorithm named as hybridization.The standard hybrid application is to employ PSO with other stochastic based technique like GA or other evolutionary computing algorithms.Furthermore, other approach is to utilize the gradient-based approach as incorporated segment.Restructuring the equations (2.2) and (2.3) has the significant importance in the improvements of the algorithm.In such rearranging or manipulations and updation of the equation of velocity is expanded [5] with an additional factor or reducing it rely on the technique.
Standard PSO parameters ω , c 1 and c 2 can be used as constant, periodically, chaotic, random, adaptive, linear changeable or nonlinear relying on time or other concerns like cost function and velocity measures.Neighborhood topologies created by choosing the position vector of particle from Pi to Pg .In standard algorithm of PSO, individual particle is pulled towards the particle's personal best and global best.
Consequently, none else personal particle's best position vector will take the appealing impact on a single particle itself and also best position vector also will not be modified at each epoch.At present, various neighborhood topologies are presented [6], population of swarm is also explored in many different dimensions.
By selecting swarm size dynamically might be encouraging to find out the solution few optimization real world problems [7].Motivation from other population based search algorithm like GA, opposition based learning techniques, simulating annealing are the famous approaches used for the fundamental working enhancement of the standard PSO.In addition, GA common operator's selection, crossover, elitism and mutation can incorporated in PSO architecture [8].
Mutation is mainly used in GA operators due to shortcomings of PSO [9], because of deficient diversity that drives the swarm particles to approach the position discovered yet in the swarm cause local minima.Lack of diversity improving techniques, majority of optimization approaches cannot be effectively explore or exploit the given search space.Mutation operator set up the new particles by altering the current particle in the swarm [10], so incorporating diversity in the swarm and probably stopping stagnation of exploration for local minima.
Consequently, the mutation application technique draws some new modification to the algorithm like condition for mutation application, the placement of mutation is to apply and choose the different distribution sequences.In order to find the criterion for mutation applications, description of diversity threshold [11], similarity and mutation probability ration may be used.Similarly, for viable random distribution, sequences are Cauchy distribution, Chaotic distribution, Beta distribution and Gaussian distribution.
Higashi and Iba [12] implemented initial practice of mutated operator by introducing the random number based on Gaussian distribution for changing the particle dimension.Likewise Higashi, a new mutation operator is proposed by Stacey et al [13] by mutilating the particle using a number drawn from cauchy distribution to keep the diversity.Pant et al. [14] gave a new mutation operator named as Sobol mutation operator that utilizes the quasi-random sequences to explore the search space better than random distribution.To keep the diversity in the swarm Zavala et al. [15] gave two separate perturbation operators known as C-perturbation operator and M-perturbation operator and implemented to personal best position in lieu of perturbing the position vector of particle in the swarm.
Jia et al. [16] used two mutation operator called as Chaotic mutation operator and Gaussian mutation operator.By using chaotic mutation operator, global searching was performed while to solve the issue of local exploitation, Gaussian mutation was integrated into PSO.To avoid from local minima, Wu et al. [17] proposed a novel mutation named as cloud mutation having the features of randomness; and keeping the capability of standard cloud model.
Chen et al. [18] defined a novel mutation operator named as adaptive mutation.In this variant, they contain the potential particle for mutation around the global best discovered by either particle in given search space during preliminary epochs.
Liu et al. [19] defined new mutation operator with names Chaotic PSO by integrating chaos in PSO having adaptive inertia weight.To avoid from premature convergence and keeping the diversity in swarm, Yang et al. [20] proposed a new mutation operator that uses the chaotic probability into the algorithm having inertia weight linear decreasing.
Liu et al. [21] defined an improved version of PSO focused on the concept of collectivity in which resemblance focus on current global best position vector and a particle in the swarm.Biao et al. [22] proposed PSO based on fast position convergence, in order to prevent from unwanted epochs in each local optima, and it is used only when required.
According to [23], by proposing the intelligent PSO called (PSO-IM) based on intelligent mutation.PSO-IM includes two types of mutation, uniform mutation and non uniform mutation.Tournament selection strategy is adopted to select particles randomly in each tournament for mutation operation while mutation probability (p m ) is controlled dynamically with fuzzy controllers.The process of tournament selection continues until a predetermined number of individuals selected.Uniform mutation replaced selected particles by a random number while non uniform mutation applied on rest of particles to overcome local minima.The performance of PSO-IM was tested on six popular nonlinear global optimization problems and four nonlinear reliability problems .PSO-IM outperforms the original PSO in all test cases.K. Wang and F. Li [24] brought dynamic chaotic behavior in PSO called (dcmPSO) to gain global exploration at the start of iteration and local exploitation at the end of the iteration.The logistic map turns the mutation process into chaotic state.At the end of iteration a temporary leader is found.If the temporary leader does not improved for a predefined constant number.Than chaotic state turns on to update temporary leader.The proposed technique proved its effectiveness on Park-Ramirez bioreactor dynamic problem.
The mutation operator gives the diversity ability in the swarm.Thus, for mutation application, mutation operator types and their application technique is the major decision portion.Besides the proposing of new mutation operator to avoid from local minima, the researchers put very limited attempts to examine how to use these new deigned mutated operators through the PSO procedure and find which kind of diversity in the swarm should be available.

III. PARTICLE SWARM OPTIMIZATION
PSO is a relatively new metaheuristic optimization search algorithm that uses the pool of best solutions to find the optimum solution.The search of optimum solution is managed by adopting the social behavior of bees, herds of animal and bird flock [25].Considering the ants' colonies, bee swarm, flock of birds, animal herds and school of fish revealed that collective venture of a bunch is normally more productive than single effort.Each single entity inside a group has a specific ability to achieve the goal.During working in a group, action of a candidate led not only by the candidate's understanding to accomplish the goal but also through the social action.The entire candidates inside a group share the experience by following the common goal, and each individual discover not only from its own experience as well as from experience of its neighbors.This accelerates the searching process considerably fast.This kind of social interaction was the origination of the PSO algorithm, elaborated in the paper.PSO works on bunch of candidates.Each candidate termed as a particle which depicts a solution for the optimization problem.For n dimensional problem, a particle depicted by n-dimensional vector, x represents a particle position.Each particle has a fitness value that shows the worth of individual's solution representation.Swarm's particle moves in n dimension given search space.Beside the position vector in search space, each particle has velocity vector that finds the step size and direction of particle motion.Social communication emulated by making neighbors inside the swarm.Each particle saves its own personal best position discovered until now and can inquire neighbor particles for the best position as found by the neighboring particles so far.PSO neighboring swarm information sharing techniques have been discussed in the literature [26].
The architecture and size of neighbor finds the method where information shared between the particles.PSO seeks for an optimum solution by moving the particles over the n dimensional search space.For each step k, position vector of particle X k id updated by adding the velocity vector X k+1 id of the particle to the prior position vector.
The velocity vector of the particle finds out the step size and direction.The velocity equation is given below: Where acceleration coefficients c 1 and c 2 applied to measure the effect of cognitive part (second term of equation 2.2) and social component (third term of equation 2.2); r 1 and r 2 are random numbers vectors, where each part taken from uniform distribution in the range between zero and one.In each iteration value of r 1 and r 2 changed.p id is particle personal best position acquired by the particle 'y' yet; where as p gd is the best global position discovered by any particle in the neighbor of particles.
Although in the original form of PSO, none method exists to restrain the velocity.It causes the feeble nature of algorithm, particularly in the area of local minima.In the back adaptations of PSO, this insufficiency was handled by incorporating two new parameters, named as inertia weight proposed by Shi and Eberhart [27] and constriction factor (x) presented by Clerc [28].Therefore, each particle is approaching to the best position confronted by itself until now, along with entire best position found by the neighborhood particles, so far.A maximum velocity V ( max) occasionally adjusted to restrict particle velocity in each dimension of the search space.Velocity clamping performed to stop the particles from traveling the search space quickly, since extremely big steps stop particles from exploiting good areas of the search space.V ( max) is implemented by stopping V i d k in each dimension space [29].Major advancement in PSO is the addition of inertia weight term to restrain the influence of value of older velocity on new velocity is given below: Where the term ω is named as inertia weight; two positive constants c 1 and c 2 are cognitive and social parameter, respectively.Currently, Clerc [5] inducted one more parameter named as constriction factor, which may help to anchor the convergence.The constriction factor model clarifies the system by choosing ω , c 1 and c 2 for the guaranteed convergence.By selecting these parameters values accurately , the velocities of all particles are selected in the range [−V ( max), V ( max)].Eberhart and Shi investigate the performance of PSO using velocity clamping V ( max) with constriction factor.The experimental outcomes showed that incorporation of constriction factor [30] boosts the convergence of algorithm.When constriction factor is examined on test benchmark problems, it remained unsuccessful to attain the certain threshold error for the given problem within allotted stipulated number of epochs.Subsequently it was established that as the particles remained away from the given search space, the constriction factor unsuccessful to attain the given number of epochs.After setting the velocity clamping to constriction factor, the performance was enhanced for all benchmark problems.
After adjusting the values of c 1 and c 2 in the equation, it might provide for accelerating the convergence of the algorithm.Choosing the default values c 1 = c 2 = 2 was suggested but the simulation results shows that different combination according to the problem nature may give better performance.Latest work [10] reveals that it could be still best to select a smaller social parameter than cognitive parameter.r 1 andr 2 are random numbers vectors where each part taken from uniform distribution in the range U(0,1), has been utilized to keep the diversity.Magnitude of the velocities managed by the constriction parameter factor x likeV ( max) parameter practiced in the initial version of PSO.The algorithm gives the quicker convergence speed, when x and V ( max) are collectively used.Three primarily operators are normally used in EA approaches: the selection, the recombination and the mutation operator.PSO does not have a recombination operator.Stochastic improvement of a particle towards its past best position, notwithstanding the best particle of the swarm [31], mimics the recombination method in EA.In PSO, transfer of information occur only among the particles of the swarm by their own experience and the experience of best particle in the population, rather choosing the fitness from elite "parent" to the off spring in GA's.Furthermore, in PSO position updating vector matches the mutation operator in GA.PSO relate to the kind of EAs that does not practice the concept of "survival of fittest".It does not use the selection procedure directly.Hence, particle with less fitness can exist during optimization process and possible visit any point of the search space in the swarm.Pseudo code of standard PSO is presented in the Fig. 1: IV.PROPOSED METHODOLOGY SCHEME PSO is an evolutionary intelligent searching approach relies on the population.It is used to find the optimum solution in the search space based on collective cooperation and contest between the groups.Like other swarm intelligence techniques, PSO algorithm faces the problem of premature convergence.
In PSO algorithm, nature of particle is determined by its global best location and previous best location of the particle discovered yet [32].When these best particles trapped in local minima, the current particle could be fallen in local minima.To recover from this phenomenon, chi-square mutation operator is introduced.If particle is fallen in local minima, proposed mutation operator will support it, recover it from local minima and move this particle to another location far from the local minima.In this proposed technique, global best particle is mutated by Chi -Square.By using chi-square in PSO, PSO takes a long jump to escape from local minima.The chi square distribution is one of the mathematical distributions which have large usage in statistical work.The term chisquare (pronounced as ch).The Greek letter X is used to represent this distribution.The probability density function (pdf) of the chi-squared distribution is given in Equation 4.
in the Eq 4 (∂) is fixed as 0.1 and Zx refereed as numerical object .Normally in population, based meta heuristic search algorithm like PSO, working performance relies on population initialization, which finds the succeeding methodology evaluations.Standard PSO uses the random uniform distribution numbers for initialization of population [33].
Because of this and the issues of large dimensions, search space exploration is ineffective and non-distribution of swarm particles may take place.It causes the slow convergence and creates the process to escape from local minima.Recently, to avoid these shortcomings, different population initialization methodologies have been developed and utilized in various research fields to prevent from premature convergence in metaheuristic algorithms and enhance the efficient exploration in search space.
Quasi random sequences are not as much random than the pseudo random sequences, however these are more strong for computational techniques who rely on creation of the random numbers.A few famous quasi random numbers are Sobol, Halton, Torus, Faure and Vander Corput.
Such sequences have been employed for initialization.A simulation outcome depicts a remarkable progress over standard PSO that employed uniform distribution.The experimental results revealed that using QRS for initiation of population enhance the performance of meta heuristic algorithms.The flow chart of the proposed technique is presented in Fig. 2. The prime phases of proposed technique are given in Algorithm 1.

V. EXPERIMENTAL RESULTS
The proposed chi-square mutated PSO (Chi-Square PSO) is simulated in C++ and applied on computer with 2.3 GHz Core (M) 2 Duo CPU processor .In order to measure the execution of the proposed chi-square PSO algorithm, a group of benchmark functions has been utilized to do the comparison with many other improved PSO techniques with traditional PSO, Adaptive PSO and different initialization techniques.Eight non-linear test functions are chosen here to examine the optimization outcomes of proposed Chi-Square PSO that are normally applied to investigate the performance of any technique.

A. Experimental Setup
The parameters for simulation used as c1=c2=1.45,inertia weight w is used in the interval [0.9,0.4] and swarm size is 20.For all the simulation, the function dimensions are D=10, 20 and 30 and maximum number of epochs is 3000.For fair comparison, all techniques apply similar parameters.In order to check the performance of each technique, all algorithms tested for 30 runs.

B. Benchmark Functions
This segment contains the eight benchmark functions applied to test the performance evaluation of the proposed algorithm.List of these functions is available in the Table I, D shows the dimensionality of the problem, S represents the interval of the variables and fmin denotes the common global optimum minimum value.

C. Discussion
The main objective is to examine the progress produced by the proposed technique.For a fair comparison, the performance of following PSO variants is tested: Standard PSO, PSO with adaption mutation (AMPSO), Chi-Square PSO and also compared the results Chi-Square PSO with Sobol initialization and Chi-Square PSO with Halton initialization using mutation operator and without using the mutation operator.From Fig. 3 to 8, it is shown that not only the Chi-Square PSO provide fast convergence speed over AMPSO and from standard PSO.Simulation Results depicts that proposed technique improve the exploration capability but also provide fast convergence and achieve the global diversities and global optima.Table II shows the comparative results.

VI. CONCLUSION
This paper introduces a new approach of PSO algorithm by proposing a Chi-Square mutation operator and using two different quasi random initialization techniques have been joined with proposed PSO and employed on function optimization problems.The proposed mutation strategy maintains the diversity of the swarm and improves the global searching capability.The simulation results show that the proposed mutated PSO has better convergence accuracy and can escape Algorithm 1 Proposed PSO Step 5:Updating the each particle velocity using the equation 12 : Step 6:Updating the each particle position using the equation 13: Step 7:Apply chi square mutation on the global best particle by using Eq (4).14: If mutated particle is better than current Pi, substitute the current particle Pi with mutated particle; else retain the particle Pi unchanged 15: Step 8:If the stopping criteria met, stop the epoch process; else go to step 2 from premature convergence successfully and compared with other recognized variants of PSO.The future work is to theoretically examine its effects and employ it some real world complex optimization problems.For future research work, it will be exciting to focus on the proposed approach to many realworld engineering applications.Furthermore, it is interestingly important to implement the proposed technique for engineering optimization problem to enhance its practicability and rightness.

1 :
Step 1: Initialization 2: Set epoch number I=0, swarm population size NP, the particles dimensions D in the swarm , wmax, wmin, C1, C2, For each particle Pi in the swarm 3: Step 1.1: Initialize the Particle position Xi 4: Step 1.2: Initialize the Particle velocity Vi 5: Step 1.3: Calculate the fitness value fi 6: Step 1.4: Set global best position gbest with the particle with best-evaluated fitness function computed value in the swarm. 7: Step 1.5: Set the personal best location pbest of each particle in the swarm as Pi= Xi. 8: Step 2:Compare the value of current particle's fitness in the swarm and its previous best location pbest .If the particle fitness is better than pbest, then substitute the pbest with its current fitness; else retain the particle Pi unchanged 9: Step 3:Compare the value of current particle's fitness in the swarm and its global best location gbest .If the particle fitness is belter than gbest, then substitute the global best position gbest with its current fitness ; else retain the particle Pi unchanged 10: Step 4:Execute chi square mutation to each particle 11: