A Hybrid Algorithm Based on Firefly Algorithm and Differential Evolution for Global Optimization

In this paper, a new and an effective combination of two metaheuristic algorithms, namely Firefly Algorithm and the Differential evolution, has been proposed. This hybridization called as HFADE, consists of two phases of Differential Evolution (DE) and Firefly Algorithm (FA). Firefly algorithm is the natureinspired algorithm which has its roots in the light intensity attraction process of firefly in the nature. Differential evolution is an Evolutionary Algorithm that uses the evolutionary operators like selection, recombination and mutation. FA and DE together are effective and powerful algorithms but FA algorithm depends on random directions for search which led into retardation in finding the best solution and DE needs more iteration to find proper solution. As a result, this proposed method has been designed to cover each algorithm deficiencies so as to make them more suitable for optimization in real world domain. To obtain the required results, the experiment on a set of benchmark functions was performed and findings showed that HFADE is a more preferable and effective method in solving the highdimensional functions. Keywords—Differential Evolution; Firefly Algorithm; Global optimization; Hybrid algorithm


INTRODUCTION
One of the affective methods in finding the best solution in numerical problems is the Optimization technique.In optimization, only a few solutions are considered the best which are called as the goal.Classical optimization techniques have some deficiencies on solving the complex optimization problems.These deficiencies are primarily interdependent on their inherent search systems.These classical optimization methods are strongly under effects of choosing proper objectives, constraints functions and type of variables.They also do not grant a universal result approach that can be used to solve problems where various type of variables, objective and constraint functions, are used [1].For covering these deficiencies, a new method with the name of metaheuristic was designed, which is mainly originated from artificial intelligence research that developed by researchers [2].A metaheuristic is an algorithm designed for solving the various types of hard optimization problems without having to fully accommodate to each problem.The Greek word meta indicates that these methods are higher-level heuristics.The primary features of metaheurisitc methods are as follows: they are nature-inspired (meaning that they have originated from nature physics, behavior and etc); stochastic components are one of the inseparable parts of these methods (involving random variables); they arent gradient base method and dont use them; at the beginning of program, they have several parameters which needs to adjusts properly.Metaheuristic algorithms combine various intelligent procedures and guide basic heuristic methods [3].These algorithms are inspired from different things such as natural phenomena, natural selections and social behaviors and applied in solving the optimization problems.Examples of the recently metaheurtistc algorithms are Vortex search [4], WOA (whale optimization algorithm) [5], MBA (mine blast algorithm) [6], WCA (water cycle algorithm) [10], and SFS (stochastic fractal search) [8].
The Firefly algorithm (FA) [9] is one of the natureinspired algorithms presented to perform global optimization in complex search spaces.In fact, it uses the act of firefly in nature and simulates behave of attraction to the flashing lights of fireflies.
One of the population based metaheuristic algorithms is the Differential evolution algorithm (DE) which is modeled on Darwins evolutionary principle of Survival of the Fittest [10].DE, like the Genetic algorithms, benefits from the Natural Selection Theory and uses its operators like the crossover, mutation and selection to create the new population for the next generation.Over the last decades, experiments on the DE algorithm have proven that it is the simplest algorithm which shows the best performance in metaheuristic algorithm for global optimization and in real parameter optimization.The most important difference between GA and DE is that DE uses distance and direction information from the current population to guide the search process.
Hybridization of DE with other algorithms has been investigated in many studies.DE-VNS [11]is a new type of hybrid method which combines two well-known metaheuristic approaches: Differential Evolution (DE) and Variable Neighborhood Search (VNS) [12],which has, in the last decade, attracted considerable attention in both academic circles and among practitioners.A promising new natureinspired algorithm known as FA was recently proposed and has gained more attention in the research literature.The ACOFA [13] is the new hybridization for FA and ACO [14] algorithm.This hybrid algorithm has been designed to solve unconstrained optimization problems and FA works as a local search to refine the positions found by the ants.In this paper, we will combine DE and FA global optimization algorithms, and propose the novel hybrid algorithm based on these algorithms which are jointly called as HFADE.As DE has Operators like crossover and mutation, this could provide www.ijacsa.thesai.orgmore variant population for FA which could help in finding more lighter fireflys algorithms.
In the real world, many problems have been porposed and optimization problems are one of them [15].The optimization problems are single or multi-objective.The multi-objective is the problem with more than one objective function (m¿1) and single objective is a problem with one objective function (m=1).The main goal in this procedure is to seek the global minimum or maximum.The function may have more than one minimum or maximum which is called as the local, but only one of them is the global maximum or minimum.The point x* is the global minimum if f(x*)≤f(x) for all the x in the searching space S. Optimization problem may consist of one or more mathematical functions which need to be optimized.The general form of the optimization problem is indicated in Eq. (1).
Where n is the decision variables, m is the number of objectives, x is decision vector and S is searching space.If the problem has one objective function (m=1), then it should be indicated as Eq. ( 2).
The rest of the paper is organized as follows: Section 2 illustrates the DE and FA algorithms, and section 3 discusses the HFADE algorithm, its parameters and boundary control.Section 4 presents 26 benchmark test functions applied for the experiments.Finally, the last section presents the concluding remarks.

II. FIREFLY ALGORITHM AND DIFFERENTIAL EVOLUTION
A. Firefly Algorithm (FA) Firefly algorithm was introduced by X.S Yang for the first time in 2007 [9].This algorithm has been inspired from the fireflys behavior and its attraction to the light.It has more similarity to PSO [16]and therefore implementation for this algorithm is much easier.Flashing insects at nights are fireflies and they gather together when they have started the flashing.Each firefly releases small rhythmic light flashes which has a light intensity attraction I and this light will decrease by distance r increment.So, each firefly attracts to the other firefly which is lighter and nearer to them.Intensity of light in the firefly depicts its fitness value, meaning the lighter the Firefly, the more fitness one has over the other.This light-intensity attractiveness of firefly is demonstrated by I in the firefly algorithm.

1) Attractiveness and Light Intensity:
Each firefly has position in searching space and the light intensity for this firefly in distance r is I(r) which is fitness value for distance r.The light intensity decrease base on the square of distance.So, the I (r) varies according to the well-known inverse square law.

()
Fireflies attractiveness is proportional to the I(r) seen by surrounding fireflies can be defined as where γ is the light absorption coefficient.

2) Distance:
The distance between fireflies i and j (r i,j ) is computed by euclidean distance formulation for d dimensional position x .This distance calculated as follow: Movement: The new position for firefly calculated by Eq. ( 6).In this equation first term is previous position and second term is used for determining the attractiveness() of a firefly (attractive firefly), towards the attractive neighboring fireflies and third term causes movement in random direction toward the brighter firefly xi.
if the firefly i is not brighter than firefly 'j' then the algorithm will perform random walk with following formulation: where the coefficient α is a randomization variable, and 'rand' is a random real number between interval [0 1].www.ijacsa.thesai.orgFirefly algorithm is explained as follows:

B. Differential Evolution (DE)
Most of the metaheuristic algorithms start the searching space with initial population.These methods are populationbased metaheuristic algorithms and DE is one of them [10].The DE is simple to implement and requires a minimum parameter to be adjusted so as to make it the best for combinatorial optimization.It requires to be adjusted based on three parameters which are F, CR and N (population size).Parameter F is the weighting coefficient that is used to generate new trial solutions and CR is the crossover probability which is used to specify a rate of crossover.It has been argued that the DE algorithm is highly under the effects of choosing proper values for the CR and F, and hence, changing the proper amounts of F and CR during algorithm execution can boost its efficiency.The mutation plays the most important role in the DE algorithm, and as a result, it is one of the main parts of the DE.The mutation equation is as follows: here a, b and c are random selected agents from population and i is the index from dimension d.The Xnew is the new agent which is created from the mutation.The DE algorithm is described as follows:  A HYBRID ALGORITHM BASED ON FIREFLY ALGORITHM AND DIFFERENTIAL EVOLUTION The main bulk of research on the FA algorithm has been demonstrated that, in FA algorithm, each firefly moves to the brighter one and when it does not find any brighter firefly, it performs random walk.The random walk is a simple and weak operator that has been performed in FA.Then, this can be changed, and instead, one can use another strong and better operator so as to make the variation in the desired firefly.To circumvent this deficiency, the hybrid algorithm of FA and DE, hence forth, is called (HFADE) has been proposed.The DE algorithm performs the mutation and crossover on the one firefly which couldnt find a brighter firefly.When the supposed firefly couldnt find a brighter one around itself, then, it might be assumed that the supposed firefly is the local best.So, the DE Crossover and mutation operators apply variation to that local best which could help in find another better place around the firefly under question, and thus, avoiding the trap in local solutions.The general steps of (HFADE) algorithm are as follows: First, The initial population ought to be created, and then, each firefly should be allocated the random position and the cost (light intensity) for that position should also be computed.The next step is the process of firefly algorithm.In this step, based on the value of cost (light intensity) in each firefly, if the preconditions of moving to the another firefly is not satisfied, then, it will be used to perform the process of updating in firefly algorithm by Eq. ( 6).Otherwise, we go to the next step which performs the process of the DE algorithm.In DE algorithm phase, the mutation and crossover operators are applied on those fireflies which couldnt find their brighter counterparts and the conditions didnt apply to them.The new firefly which has been produced by the DE is compared with the previous one.If the cost (light-intensity) for the newly-produced firefly is better than the previous one, then, it will be swapped with the previous one, and therefore, the new firefly can occupy a position and if the new firefly is better than the global best, then, the global best will also be swapped with the new firefly.Finally, in the last step, if the termination criterion is satisfied, then, the global best firefly should be considered the output.Otherwise, the next iteration will be started from firefly algorithm again.Figure 1 shows the simple flowchart for the proposed algorithm (HFADE).As can be seen in this figure, the (HFADE) is consisted of two main phases, and in each phase, it runs in parallel forms.The proposed hybrid (HFADE) algorithm is as follows:

2) Convergence analysis
For analyzing convergenc of the metaheursitc algorithm, Markov chain Monte Carlo method is the one of the preferrable methods for this task [17].Most metaheuristic algorithms can be sighted in the framework of Markov chain from statically viewpoint.Now if look at the proposed hybrid method closely using the framework of Markov chain Monte ) do 11: for eachoffireflyi(X i ) do 12: for eachoffireflyj(X i ) do 13: if I(X i ) < I(X j ) then 14: CalculatedistanceX i andX j byEq.( 5  Theorem 1. HFADE algorithm converges to its globally best solution with probability 1. Proof: Let P 0 (g) = P{ω(N(g)) = 0} then the probability due to the Bayesian conditional probability of P 0 (g + 1) is Since the best solution replace with previous one in memory, expression P{ω(N(g true because of the HFADE algorithm by two main phases FA and DE store the best solution. . such that, 0 ≤ P 0 (g + 1) ≤ (1 − Γ) × P 0 (g).Hence, Given that g lim →∞ (1 − Γ) g+1 = 0 and 0 ≤ P0(0) ≤ 1.

C. Parameter adjustments and boundary control
Parameter adjustment is a non-negligible task which is required to be performed properly in order to get a better result in solving various problems.Besides, the parameter adjustment also is necessary for controlling the boundary whenever the algorithm finds a new solution [18].The (HFADE) needs boundary control for a Firefly Xi, because its position is required to be in the searching space, which is a boundary between [L U], (L is the lower bound and U is the upper bound of the searching space).The method that controls the boundary is as follows: Where Min and Max are the functions that select the minimum and maximum among the input pairs, X is the input firefly position and q is the output which have been controlled in the boundary range [L U].As it was alluded to previously, The DE algorithm is a simple algorithm and enjoys two primary parameters, F and Crossover rate (CR), which is required to be adjusted properly and FA has three main parameters, Light Absorption Coefficient (gamma), Attraction Coefficient Base Value (beta0), and Mutation Coefficient (alpha).So, the combination of two algorithms of FA and DE also enjoys these parameters.

IV. BENCHMARK TEST FUNCTIONS
In this paper, the proposed algorithm has been evaluated with a set of benchmark functions which are a subgroup of unimodal and multimodal functions.These functions have various dimensions such as 2, 4, 10 and 30.Tables 2 highlights a multimodal test functions and Table 1 shows unimdal test functions.Multimodal test functions have many local minimums, and therefore, they are hard to be solved simply because of the trapping in local solutions.The (HFADE) has been evaluated by these functions and search the global minimum for them.In Table 3, the results for algorithms GA [19], DE , PSO, BA [20], PBA [21] and FA that compared with HFADE.Conditions for experiment and paramter settings for these algorithms is explained in reference [21].The parameter settings for HFADE are as follows: n=20 (populationsize), γ=2 , β 0 =2, α=0.2, CR=0.2 and F ∈ [0.2 0.8].The experiment has been performed on the computer with following features: CPU 2.1 GHZ, Ram 8 GB and Matlab 2013 running on computer with windows 10.The NFE= 500,000 (number of function evalution) was set as Stopping criteria and the values minimum than 1E-12, presented as 0 same as other methods.The mean value and Std Dev (standard deviation) have been calculated from 30 independent runs.In Table 3, the HFADE found the minimum results for most of the functions with best standard deviations.It only did not reach the minimum average value in functions Quartic (F12), Dixon-price (F8), Michalewicz 10 (F25) and schewefel 1.2 (F7).The results of Friedman non-parameteric test [22] are also presented in this table and HFADE could rank the best with value 3.15.The low p-value indicates that the results are remarkably different with each other and figure 3    DE.Also, the test of normality of Kolmogorov-Smirnova and Shapiro-Wilk [22] has been performed for the three functions of F7, F12 and F25 which are hard to be solved, and here, standard deviation is not zero for them.Table 4 presents the results for this test and the p-value, df (degree of freedom), and the statistics for this test are also presented.Accordingly, the p-value which is higher than the significant =0.05 is considered a normal distribution and the lower than that value is supposed to be an abnormal distribution.Based on the pvalue and the test of normality of Kolmogorov-Smirnova, the function of F7 result is normal and other functions have the abnormal distribution.Fig. 4 shows the normal and abnormal distributions, the histogram, QQ-plot and Box-plot for the two functions, F7 and F12.F7 is a normal distribution and F12 is considered abnormal.As can be inferred, in normal distributions, the results are in one diagonal line for QQ-plot while in abnormal distributions, this fact does not hold true

V. CONCLUSION
The FA is, arguably, one of the most efficient natureinspired metaheuristic algorithms, which has outperformed most of the algorithms in solving the various optimizing numerical problems.Furthermore, one of the practical metaheuristic algorithms which have been used most widely in the optimization is the DE.The DE algorithm is a simple and practical algorithm which is more amenable to combination with others.In the current study, the FA and DE algorithms are combined in order to design a new hybrid method for seeking the global solution.The proposed hybrid algorithm begins to search from the FA algorithm if the precondition for entering the FA algorithm is not satisfied the process is passed to the DE which makes the alteration with crossover and mutation operators on the agent of population.The experiment based on benchmark functions and non-parametric ranking shows that the proposed hybrid is more dominant and competent than other famous algorithms.Moreover, based on the results of the test of normality and convergence, this proposed hybrid algorithm mostly has abnormal distributions for results and converges to the optimum solutions in minimum number of iterations.

Definition 2 .
where X is probable solution and f is fitness function.The number of best fireflies in firefly population is shown by λ(N) :| N ∩ O * |.Algorithm convergence with probability 1 to the best if this condition is true :g lim →∞ P{ω(N(g)) ≥ 1 | N(0) = N 0 } = 1,where g indicate generation number and N 0 is random initial population.
depicts the results for this test with a bar diagram.For analyzing the procedure of convergence in (HFADE), figure 2 has been presented.This figure shows the Convergence diagram for functions F19, F25, F18 and F20 in algorithms FA, DE and (HFADE).As it stands, HFADE has reached the desired minimum with minimum number of iteration and faster against the other algorithms FA.

TABLE I .
UNIMODAL TEST FUNCTIONS (D: DIMENSIONS)

TABLE III .
HFADE COMPARISON WITH GA, DE, PSO, BA, PBA AND FA (UNIMODAL FUNCTION SET), BOLD VALUES REPRESENT THE BEST

TABLE IV .
TEST OF NORMALITY KOLMOGOROV-SMIRNOVA AND SHAPIRO-WILK FOR FUNCTIONS F7,F25 AND F12